Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251028 02:59:48.213462 7786 test_util.cc:276] Using random seed: 897053377
2025-10-28T02:59:48Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-28T02:59:48Z Disabled control of system clock
I20251028 02:59:48.230504 7786 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
/tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.190:39091
--webserver_interface=127.7.154.190
--webserver_port=0
--builtin_ntp_servers=127.7.154.148:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20251028 02:59:48.342559 7794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251028 02:59:48.342835 7794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251028 02:59:48.342875 7794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251028 02:59:48.346274 7794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251028 02:59:48.346355 7794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251028 02:59:48.346377 7794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251028 02:59:48.346402 7794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251028 02:59:48.350458 7794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.154.148:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.190:39091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.154.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 60f5e5267b92c39485a66121d3ce3cc7ef57b0e0
build type DEBUG
built by None at 28 Oct 2025 02:43:12 UTC on 24a791456cd2
build id 8750
I20251028 02:59:48.351758 7794 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251028 02:59:48.352993 7794 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 02:59:48.359565 7799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 02:59:48.359553 7800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:48.359647 7794 server_base.cc:1047] running on GCE node
W20251028 02:59:48.359525 7802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:48.360333 7794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251028 02:59:48.361357 7794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251028 02:59:48.362517 7794 hybrid_clock.cc:648] HybridClock initialized: now 1761620388362493 us; error 41 us; skew 500 ppm
I20251028 02:59:48.364533 7794 webserver.cc:492] Webserver started at http://127.7.154.190:40487/ using document root <none> and password file <none>
I20251028 02:59:48.365180 7794 fs_manager.cc:362] Metadata directory not provided
I20251028 02:59:48.365243 7794 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 02:59:48.365463 7794 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 02:59:48.367197 7794 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data/instance:
uuid: "ac9bf5a198d042698c9f030a59228865"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
I20251028 02:59:48.367746 7794 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal/instance:
uuid: "ac9bf5a198d042698c9f030a59228865"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
I20251028 02:59:48.371479 7794 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20251028 02:59:48.373812 7808 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:48.374927 7794 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:48.375058 7794 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data,/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal
uuid: "ac9bf5a198d042698c9f030a59228865"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
I20251028 02:59:48.375156 7794 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 02:59:48.396095 7794 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 02:59:48.396879 7794 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251028 02:59:48.397073 7794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 02:59:48.404698 7860 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.154.190:39091 every 8 connection(s)
I20251028 02:59:48.404709 7794 rpc_server.cc:307] RPC server started. Bound to: 127.7.154.190:39091
I20251028 02:59:48.405929 7794 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data/info.pb
I20251028 02:59:48.406972 7786 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu as pid 7794
I20251028 02:59:48.407133 7786 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal/instance
I20251028 02:59:48.408569 7786 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
/tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.189:45473
--webserver_interface=127.7.154.189
--webserver_port=0
--builtin_ntp_servers=127.7.154.148:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20251028 02:59:48.409262 7861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:48.413120 7861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } has no permanent_uuid. Determining permanent_uuid...
I20251028 02:59:48.424669 7861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } has no permanent_uuid. Determining permanent_uuid...
W20251028 02:59:48.425509 7809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111)
W20251028 02:59:48.426462 7861 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.7.154.189:45473: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111)
I20251028 02:59:48.475728 7861 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } attempt: 1
W20251028 02:59:48.477327 7861 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.7.154.189:45473: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111)
W20251028 02:59:48.523108 7863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251028 02:59:48.523404 7863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251028 02:59:48.523480 7863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251028 02:59:48.526897 7863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251028 02:59:48.526979 7863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251028 02:59:48.527015 7863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251028 02:59:48.527052 7863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251028 02:59:48.531615 7863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.154.148:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.189:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.154.189
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 60f5e5267b92c39485a66121d3ce3cc7ef57b0e0
build type DEBUG
built by None at 28 Oct 2025 02:43:12 UTC on 24a791456cd2
build id 8750
I20251028 02:59:48.532924 7863 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251028 02:59:48.534330 7863 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 02:59:48.540575 7874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 02:59:48.540532 7872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 02:59:48.540532 7871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:48.540906 7863 server_base.cc:1047] running on GCE node
I20251028 02:59:48.541428 7863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251028 02:59:48.542426 7863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251028 02:59:48.543601 7863 hybrid_clock.cc:648] HybridClock initialized: now 1761620388543584 us; error 32 us; skew 500 ppm
I20251028 02:59:48.545604 7863 webserver.cc:492] Webserver started at http://127.7.154.189:36469/ using document root <none> and password file <none>
I20251028 02:59:48.545572 7861 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } attempt: 2
I20251028 02:59:48.546357 7863 fs_manager.cc:362] Metadata directory not provided
I20251028 02:59:48.546452 7863 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 02:59:48.546690 7863 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
W20251028 02:59:48.547082 7861 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.7.154.189:45473: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111)
I20251028 02:59:48.548722 7863 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/instance:
uuid: "aac264d6d2274d3283bd3251ab77e583"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
I20251028 02:59:48.549252 7863 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal/instance:
uuid: "aac264d6d2274d3283bd3251ab77e583"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
I20251028 02:59:48.553022 7863 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20251028 02:59:48.555500 7880 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:48.556619 7863 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20251028 02:59:48.556748 7863 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data,/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
uuid: "aac264d6d2274d3283bd3251ab77e583"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
I20251028 02:59:48.556850 7863 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 02:59:48.582870 7863 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 02:59:48.583909 7863 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251028 02:59:48.584136 7863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 02:59:48.591357 7932 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.154.189:45473 every 8 connection(s)
I20251028 02:59:48.591378 7863 rpc_server.cc:307] RPC server started. Bound to: 127.7.154.189:45473
I20251028 02:59:48.592502 7863 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/info.pb
I20251028 02:59:48.595124 7786 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu as pid 7863
I20251028 02:59:48.595213 7786 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal/instance
I20251028 02:59:48.595534 7933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:48.595920 7786 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
/tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.188:44237
--webserver_interface=127.7.154.188
--webserver_port=0
--builtin_ntp_servers=127.7.154.148:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20251028 02:59:48.599599 7933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } has no permanent_uuid. Determining permanent_uuid...
I20251028 02:59:48.609599 7933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } has no permanent_uuid. Determining permanent_uuid...
I20251028 02:59:48.616041 7933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } has no permanent_uuid. Determining permanent_uuid...
W20251028 02:59:48.616881 7882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.154.188:44237: connect: Connection refused (error 111)
W20251028 02:59:48.617720 7933 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.7.154.188:44237: Network error: Client connection negotiation failed: client connection to 127.7.154.188:44237: connect: Connection refused (error 111)
I20251028 02:59:48.638265 7861 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } attempt: 3
I20251028 02:59:48.642794 7861 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } has no permanent_uuid. Determining permanent_uuid...
W20251028 02:59:48.644287 7861 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.7.154.188:44237: Network error: Client connection negotiation failed: client connection to 127.7.154.188:44237: connect: Connection refused (error 111)
I20251028 02:59:48.666970 7933 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } attempt: 1
W20251028 02:59:48.668447 7933 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.7.154.188:44237: Network error: Client connection negotiation failed: client connection to 127.7.154.188:44237: connect: Connection refused (error 111)
I20251028 02:59:48.675462 7861 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } attempt: 1
W20251028 02:59:48.676906 7861 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.7.154.188:44237: Network error: Client connection negotiation failed: client connection to 127.7.154.188:44237: connect: Connection refused (error 111)
W20251028 02:59:48.713335 7935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251028 02:59:48.713586 7935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251028 02:59:48.713627 7935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251028 02:59:48.717049 7935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251028 02:59:48.717132 7935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251028 02:59:48.717154 7935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251028 02:59:48.717172 7935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251028 02:59:48.721319 7935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.154.148:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.188:44237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.154.188
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 60f5e5267b92c39485a66121d3ce3cc7ef57b0e0
build type DEBUG
built by None at 28 Oct 2025 02:43:12 UTC on 24a791456cd2
build id 8750
I20251028 02:59:48.722517 7935 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251028 02:59:48.723743 7935 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 02:59:48.729364 7944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 02:59:48.729364 7946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 02:59:48.729499 7943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:48.729702 7935 server_base.cc:1047] running on GCE node
I20251028 02:59:48.730218 7935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251028 02:59:48.731226 7935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251028 02:59:48.732393 7935 hybrid_clock.cc:648] HybridClock initialized: now 1761620388732381 us; error 32 us; skew 500 ppm
I20251028 02:59:48.734303 7935 webserver.cc:492] Webserver started at http://127.7.154.188:45247/ using document root <none> and password file <none>
I20251028 02:59:48.734894 7935 fs_manager.cc:362] Metadata directory not provided
I20251028 02:59:48.734954 7935 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 02:59:48.735150 7935 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 02:59:48.736620 7933 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } attempt: 2
I20251028 02:59:48.736759 7935 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/data/instance:
uuid: "9343ca8510de4eaa9945786e054a866d"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
I20251028 02:59:48.737282 7935 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/wal/instance:
uuid: "9343ca8510de4eaa9945786e054a866d"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
W20251028 02:59:48.737969 7933 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.7.154.188:44237: Network error: Client connection negotiation failed: client connection to 127.7.154.188:44237: connect: Connection refused (error 111)
I20251028 02:59:48.741034 7935 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20251028 02:59:48.743502 7952 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:48.744618 7935 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:48.744750 7935 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/data,/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/wal
uuid: "9343ca8510de4eaa9945786e054a866d"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
I20251028 02:59:48.744851 7935 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 02:59:48.752117 7861 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } attempt: 2
W20251028 02:59:48.753582 7861 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.7.154.188:44237: Network error: Client connection negotiation failed: client connection to 127.7.154.188:44237: connect: Connection refused (error 111)
I20251028 02:59:48.756845 7935 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 02:59:48.757527 7935 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251028 02:59:48.757702 7935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 02:59:48.765177 7935 rpc_server.cc:307] RPC server started. Bound to: 127.7.154.188:44237
I20251028 02:59:48.765228 8004 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.154.188:44237 every 8 connection(s)
I20251028 02:59:48.766252 7935 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/data/info.pb
I20251028 02:59:48.769205 8005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:48.771919 7786 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu as pid 7935
I20251028 02:59:48.772014 7786 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-2/wal/instance
I20251028 02:59:48.772930 8005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } has no permanent_uuid. Determining permanent_uuid...
I20251028 02:59:48.782109 8005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } has no permanent_uuid. Determining permanent_uuid...
I20251028 02:59:48.787182 8005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } has no permanent_uuid. Determining permanent_uuid...
I20251028 02:59:48.794265 8005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Bootstrap starting.
I20251028 02:59:48.796828 8005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:48.797601 8005 log.cc:826] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Log is configured to *not* fsync() on all Append() calls
I20251028 02:59:48.799273 8005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: No bootstrap required, opened a new log
I20251028 02:59:48.802310 8005 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:48.802491 8005 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:48.802528 8005 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9343ca8510de4eaa9945786e054a866d, State: Initialized, Role: FOLLOWER
I20251028 02:59:48.802922 8005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:48.803431 8014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:48.803598 8014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:48.804275 8005 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: configured and running, proceeding with master startup.
W20251028 02:59:48.808882 8025 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251028 02:59:48.808964 8025 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251028 02:59:48.807694 7935 master_runner.cc:186] Error getting master registration for 127.7.154.190:39091: OK, instance_id { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" instance_seqno: 1761620388402568 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251028 02:59:48.810030 7935 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251028 02:59:48.829141 7933 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } attempt: 3
I20251028 02:59:48.835352 7933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Bootstrap starting.
I20251028 02:59:48.837896 7933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:48.838647 7933 log.cc:826] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Log is configured to *not* fsync() on all Append() calls
I20251028 02:59:48.840354 7933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: No bootstrap required, opened a new log
I20251028 02:59:48.843039 7933 raft_consensus.cc:359] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:48.843209 7933 raft_consensus.cc:385] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:48.843245 7933 raft_consensus.cc:740] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aac264d6d2274d3283bd3251ab77e583, State: Initialized, Role: FOLLOWER
I20251028 02:59:48.843712 7933 consensus_queue.cc:260] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:48.844190 8028 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:48.844302 8028 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:48.844890 7933 sys_catalog.cc:565] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: configured and running, proceeding with master startup.
W20251028 02:59:48.849912 8039 catalog_manager.cc:1568] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251028 02:59:48.849998 8039 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251028 02:59:48.849413 7863 master_runner.cc:186] Error getting master registration for 127.7.154.190:39091: OK, instance_id { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" instance_seqno: 1761620388402568 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251028 02:59:48.851799 7863 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251028 02:59:48.852774 7861 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } attempt: 3
I20251028 02:59:48.860517 7861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: Bootstrap starting.
I20251028 02:59:48.863567 7861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:48.864558 7861 log.cc:826] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: Log is configured to *not* fsync() on all Append() calls
I20251028 02:59:48.866801 7861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: No bootstrap required, opened a new log
I20251028 02:59:48.870646 7861 raft_consensus.cc:359] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:48.870887 7861 raft_consensus.cc:385] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:48.870949 7861 raft_consensus.cc:740] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac9bf5a198d042698c9f030a59228865, State: Initialized, Role: FOLLOWER
I20251028 02:59:48.871542 7861 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:48.872972 7861 sys_catalog.cc:565] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 02:59:48.874675 8042 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:48.874899 8042 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: This master's current role is: FOLLOWER
W20251028 02:59:48.878196 8053 catalog_manager.cc:1568] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251028 02:59:48.878355 8053 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251028 02:59:48.882921 7794 master_runner.cc:263] No leader master found from master ac9bf5a198d042698c9f030a59228865
I20251028 02:59:48.883008 7794 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20251028 02:59:48.898666 7814 tablet.cc:2378] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251028 02:59:48.955176 8028 raft_consensus.cc:493] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:48.955411 8028 raft_consensus.cc:515] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:48.956530 8028 leader_election.cc:290] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac9bf5a198d042698c9f030a59228865 (127.7.154.190:39091), 9343ca8510de4eaa9945786e054a866d (127.7.154.188:44237)
I20251028 02:59:48.957383 7979 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aac264d6d2274d3283bd3251ab77e583" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9343ca8510de4eaa9945786e054a866d" is_pre_election: true
I20251028 02:59:48.957664 7979 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aac264d6d2274d3283bd3251ab77e583 in term 0.
I20251028 02:59:48.958134 7882 leader_election.cc:304] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9343ca8510de4eaa9945786e054a866d, aac264d6d2274d3283bd3251ab77e583; no voters:
I20251028 02:59:48.959703 8028 raft_consensus.cc:2804] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:48.959790 8028 raft_consensus.cc:493] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:48.959846 8028 raft_consensus.cc:3060] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:48.959782 7835 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aac264d6d2274d3283bd3251ab77e583" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9bf5a198d042698c9f030a59228865" is_pre_election: true
I20251028 02:59:48.960083 7835 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aac264d6d2274d3283bd3251ab77e583 in term 0.
I20251028 02:59:48.961022 8028 raft_consensus.cc:515] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:48.961426 8028 leader_election.cc:290] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [CANDIDATE]: Term 1 election: Requested vote from peers ac9bf5a198d042698c9f030a59228865 (127.7.154.190:39091), 9343ca8510de4eaa9945786e054a866d (127.7.154.188:44237)
I20251028 02:59:48.961768 7835 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aac264d6d2274d3283bd3251ab77e583" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9bf5a198d042698c9f030a59228865"
I20251028 02:59:48.961920 7835 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:48.962139 7979 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aac264d6d2274d3283bd3251ab77e583" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9343ca8510de4eaa9945786e054a866d"
I20251028 02:59:48.962271 7979 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:48.963121 7835 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aac264d6d2274d3283bd3251ab77e583 in term 1.
I20251028 02:59:48.963317 7979 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aac264d6d2274d3283bd3251ab77e583 in term 1.
I20251028 02:59:48.963505 7884 leader_election.cc:304] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aac264d6d2274d3283bd3251ab77e583, ac9bf5a198d042698c9f030a59228865; no voters:
I20251028 02:59:48.964671 8028 raft_consensus.cc:2804] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:48.965360 8028 raft_consensus.cc:697] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 1 LEADER]: Becoming Leader. State: Replica: aac264d6d2274d3283bd3251ab77e583, State: Running, Role: LEADER
I20251028 02:59:48.965771 8028 consensus_queue.cc:237] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:48.967839 8028 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aac264d6d2274d3283bd3251ab77e583. Latest consensus state: current_term: 1 leader_uuid: "aac264d6d2274d3283bd3251ab77e583" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:48.968015 8028 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: This master's current role is: LEADER
I20251028 02:59:48.968427 8057 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 02:59:48.969431 8057 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 02:59:48.976763 7979 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 1 FOLLOWER]: Refusing update from remote peer aac264d6d2274d3283bd3251ab77e583: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:48.977588 8028 consensus_queue.cc:1048] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:48.981858 8014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: SysCatalogTable state changed. Reason: New leader aac264d6d2274d3283bd3251ab77e583. Latest consensus state: current_term: 1 leader_uuid: "aac264d6d2274d3283bd3251ab77e583" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:48.981999 8014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:48.983404 7835 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 1 FOLLOWER]: Refusing update from remote peer aac264d6d2274d3283bd3251ab77e583: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:48.984109 8028 consensus_queue.cc:1048] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:48.991446 8042 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aac264d6d2274d3283bd3251ab77e583. Latest consensus state: current_term: 1 leader_uuid: "aac264d6d2274d3283bd3251ab77e583" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:48.991611 8042 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:48.995546 8055 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aac264d6d2274d3283bd3251ab77e583" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:48.995721 8055 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: This master's current role is: LEADER
I20251028 02:59:48.997223 8014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aac264d6d2274d3283bd3251ab77e583" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:48.997357 8014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:48.997889 8058 mvcc.cc:204] Tried to move back new op lower bound from 7215597113239744512 to 7215597113207672832. Current Snapshot: MvccSnapshot[applied={T|T < 7215597113239744512}]
I20251028 02:59:48.998337 8059 mvcc.cc:204] Tried to move back new op lower bound from 7215597113239744512 to 7215597113207672832. Current Snapshot: MvccSnapshot[applied={T|T < 7215597113239744512}]
I20251028 02:59:49.000092 8028 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aac264d6d2274d3283bd3251ab77e583" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:49.000212 8028 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: This master's current role is: LEADER
I20251028 02:59:49.001757 8042 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aac264d6d2274d3283bd3251ab77e583" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:49.001911 8042 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:49.007508 8057 catalog_manager.cc:1357] Generated new cluster ID: a891460fa9a9449bb0459a49a032f9be
I20251028 02:59:49.007601 8057 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 02:59:49.034384 8057 catalog_manager.cc:1380] Generated new certificate authority record
I20251028 02:59:49.035887 8057 catalog_manager.cc:1514] Loading token signing keys...
I20251028 02:59:49.049296 8057 catalog_manager.cc:6022] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Generated new TSK 0
I20251028 02:59:49.050311 8057 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 02:59:49.072059 7786 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
/tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.154.129:0
--local_ip_for_outbound_sockets=127.7.154.129
--webserver_interface=127.7.154.129
--webserver_port=0
--tserver_master_addrs=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--builtin_ntp_servers=127.7.154.148:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20251028 02:59:49.187947 8066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251028 02:59:49.188195 8066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251028 02:59:49.188266 8066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251028 02:59:49.191717 8066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251028 02:59:49.191883 8066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.154.129
I20251028 02:59:49.195925 8066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.154.148:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.154.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.154.129
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--never_fsync=true
--heap_profile_path=/tmp/kudu.8066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.154.129
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 60f5e5267b92c39485a66121d3ce3cc7ef57b0e0
build type DEBUG
built by None at 28 Oct 2025 02:43:12 UTC on 24a791456cd2
build id 8750
I20251028 02:59:49.197156 8066 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251028 02:59:49.198377 8066 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 02:59:49.205319 8072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 02:59:49.205381 8074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:49.205547 8066 server_base.cc:1047] running on GCE node
W20251028 02:59:49.205319 8071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:49.206061 8066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251028 02:59:49.206661 8066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251028 02:59:49.207818 8066 hybrid_clock.cc:648] HybridClock initialized: now 1761620389207803 us; error 47 us; skew 500 ppm
I20251028 02:59:49.209702 8066 webserver.cc:492] Webserver started at http://127.7.154.129:32809/ using document root <none> and password file <none>
I20251028 02:59:49.210336 8066 fs_manager.cc:362] Metadata directory not provided
I20251028 02:59:49.210400 8066 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 02:59:49.210594 8066 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 02:59:49.212248 8066 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/data/instance:
uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
format_stamp: "Formatted at 2025-10-28 02:59:49 on dist-test-slave-97kg"
I20251028 02:59:49.212754 8066 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/instance:
uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
format_stamp: "Formatted at 2025-10-28 02:59:49 on dist-test-slave-97kg"
I20251028 02:59:49.216267 8066 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20251028 02:59:49.218735 8080 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:49.219817 8066 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251028 02:59:49.219930 8066 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal
uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
format_stamp: "Formatted at 2025-10-28 02:59:49 on dist-test-slave-97kg"
I20251028 02:59:49.220023 8066 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 02:59:49.277544 8066 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 02:59:49.278297 8066 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251028 02:59:49.278486 8066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 02:59:49.279090 8066 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 02:59:49.280288 8066 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 02:59:49.280344 8066 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:49.280403 8066 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 02:59:49.280431 8066 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:49.289971 8066 rpc_server.cc:307] RPC server started. Bound to: 127.7.154.129:46529
I20251028 02:59:49.289999 8193 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.154.129:46529 every 8 connection(s)
I20251028 02:59:49.291142 8066 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/data/info.pb
I20251028 02:59:49.299198 7786 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu as pid 8066
I20251028 02:59:49.299357 7786 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/instance
I20251028 02:59:49.301604 8194 heartbeater.cc:344] Connected to a master server at 127.7.154.188:44237
I20251028 02:59:49.301671 8196 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:49.301954 8195 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 02:59:49.302050 8195 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:49.302052 8196 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:49.302403 7786 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
/tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.154.130:0
--local_ip_for_outbound_sockets=127.7.154.130
--webserver_interface=127.7.154.130
--webserver_port=0
--tserver_master_addrs=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--builtin_ntp_servers=127.7.154.148:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20251028 02:59:49.302772 8195 heartbeater.cc:507] Master 127.7.154.190:39091 requested a full tablet report, sending...
I20251028 02:59:49.302772 8196 heartbeater.cc:507] Master 127.7.154.189:45473 requested a full tablet report, sending...
I20251028 02:59:49.302073 8194 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:49.303207 8194 heartbeater.cc:507] Master 127.7.154.188:44237 requested a full tablet report, sending...
I20251028 02:59:49.304674 7968 ts_manager.cc:194] Registered new tserver with Master: fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:49.304791 7896 ts_manager.cc:194] Registered new tserver with Master: fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:49.306542 7896 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.7.154.129:36657
I20251028 02:59:49.308363 7825 ts_manager.cc:194] Registered new tserver with Master: fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 02:59:49.420331 8203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251028 02:59:49.420662 8203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251028 02:59:49.420735 8203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251028 02:59:49.424626 8203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251028 02:59:49.424762 8203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.154.130
I20251028 02:59:49.428750 8203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.154.148:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.154.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.154.130
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--never_fsync=true
--heap_profile_path=/tmp/kudu.8203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.154.130
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 60f5e5267b92c39485a66121d3ce3cc7ef57b0e0
build type DEBUG
built by None at 28 Oct 2025 02:43:12 UTC on 24a791456cd2
build id 8750
I20251028 02:59:49.429944 8203 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251028 02:59:49.431093 8203 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 02:59:49.438112 8211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 02:59:49.438114 8208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 02:59:49.438129 8209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:49.438230 8203 server_base.cc:1047] running on GCE node
I20251028 02:59:49.438751 8203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251028 02:59:49.439385 8203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251028 02:59:49.440550 8203 hybrid_clock.cc:648] HybridClock initialized: now 1761620389440536 us; error 36 us; skew 500 ppm
I20251028 02:59:49.442409 8203 webserver.cc:492] Webserver started at http://127.7.154.130:38959/ using document root <none> and password file <none>
I20251028 02:59:49.443013 8203 fs_manager.cc:362] Metadata directory not provided
I20251028 02:59:49.443084 8203 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 02:59:49.443275 8203 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 02:59:49.444976 8203 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/data/instance:
uuid: "ccbdf478f18948d19de96574a685bfda"
format_stamp: "Formatted at 2025-10-28 02:59:49 on dist-test-slave-97kg"
I20251028 02:59:49.445636 8203 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/instance:
uuid: "ccbdf478f18948d19de96574a685bfda"
format_stamp: "Formatted at 2025-10-28 02:59:49 on dist-test-slave-97kg"
I20251028 02:59:49.449390 8203 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20251028 02:59:49.451795 8217 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:49.452852 8203 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251028 02:59:49.452961 8203 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal
uuid: "ccbdf478f18948d19de96574a685bfda"
format_stamp: "Formatted at 2025-10-28 02:59:49 on dist-test-slave-97kg"
I20251028 02:59:49.453065 8203 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 02:59:49.470007 8203 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 02:59:49.470731 8203 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251028 02:59:49.470925 8203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 02:59:49.471555 8203 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 02:59:49.472622 8203 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 02:59:49.472674 8203 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:49.472724 8203 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 02:59:49.472756 8203 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:49.482488 8203 rpc_server.cc:307] RPC server started. Bound to: 127.7.154.130:43525
I20251028 02:59:49.482519 8330 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.154.130:43525 every 8 connection(s)
I20251028 02:59:49.483577 8203 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/data/info.pb
I20251028 02:59:49.489722 7786 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu as pid 8203
I20251028 02:59:49.489869 7786 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/instance
I20251028 02:59:49.494174 7786 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
/tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.154.131:0
--local_ip_for_outbound_sockets=127.7.154.131
--webserver_interface=127.7.154.131
--webserver_port=0
--tserver_master_addrs=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--builtin_ntp_servers=127.7.154.148:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20251028 02:59:49.494732 8333 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:49.495008 8333 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:49.495074 8332 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 02:59:49.495160 8332 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:49.495357 8331 heartbeater.cc:344] Connected to a master server at 127.7.154.188:44237
I20251028 02:59:49.495420 8331 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:49.495761 8332 heartbeater.cc:507] Master 127.7.154.190:39091 requested a full tablet report, sending...
I20251028 02:59:49.495741 8331 heartbeater.cc:507] Master 127.7.154.188:44237 requested a full tablet report, sending...
I20251028 02:59:49.495746 8333 heartbeater.cc:507] Master 127.7.154.189:45473 requested a full tablet report, sending...
I20251028 02:59:49.496892 7825 ts_manager.cc:194] Registered new tserver with Master: ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:49.496917 7968 ts_manager.cc:194] Registered new tserver with Master: ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:49.498250 7896 ts_manager.cc:194] Registered new tserver with Master: ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:49.498823 7896 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.7.154.130:48907
I20251028 02:59:49.550752 8339 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aac264d6d2274d3283bd3251ab77e583)
I20251028 02:59:49.550927 8339 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:49.551939 8339 leader_election.cc:290] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ac9bf5a198d042698c9f030a59228865 (127.7.154.190:39091), aac264d6d2274d3283bd3251ab77e583 (127.7.154.189:45473)
I20251028 02:59:49.552448 7835 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9343ca8510de4eaa9945786e054a866d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ac9bf5a198d042698c9f030a59228865" is_pre_election: true
I20251028 02:59:49.552495 7907 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9343ca8510de4eaa9945786e054a866d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aac264d6d2274d3283bd3251ab77e583" is_pre_election: true
I20251028 02:59:49.553050 7953 leader_election.cc:304] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9343ca8510de4eaa9945786e054a866d; no voters: aac264d6d2274d3283bd3251ab77e583, ac9bf5a198d042698c9f030a59228865
I20251028 02:59:49.553253 8339 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251028 02:59:49.611039 8338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251028 02:59:49.611327 8338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251028 02:59:49.611407 8338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251028 02:59:49.614763 8338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251028 02:59:49.614938 8338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.154.131
I20251028 02:59:49.618901 8338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.154.148:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.154.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.154.131
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--never_fsync=true
--heap_profile_path=/tmp/kudu.8338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.154.131
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 60f5e5267b92c39485a66121d3ce3cc7ef57b0e0
build type DEBUG
built by None at 28 Oct 2025 02:43:12 UTC on 24a791456cd2
build id 8750
I20251028 02:59:49.620179 8338 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251028 02:59:49.621399 8338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 02:59:49.628235 8347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 02:59:49.628260 8344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 02:59:49.628264 8345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:49.628571 8338 server_base.cc:1047] running on GCE node
I20251028 02:59:49.628963 8338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251028 02:59:49.629510 8338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251028 02:59:49.630656 8338 hybrid_clock.cc:648] HybridClock initialized: now 1761620389630621 us; error 53 us; skew 500 ppm
I20251028 02:59:49.632498 8338 webserver.cc:492] Webserver started at http://127.7.154.131:36177/ using document root <none> and password file <none>
I20251028 02:59:49.633073 8338 fs_manager.cc:362] Metadata directory not provided
I20251028 02:59:49.633129 8338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 02:59:49.633308 8338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251028 02:59:49.634882 8338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/data/instance:
uuid: "d7e4cf694f71438298dd32a346abcf75"
format_stamp: "Formatted at 2025-10-28 02:59:49 on dist-test-slave-97kg"
I20251028 02:59:49.635394 8338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/instance:
uuid: "d7e4cf694f71438298dd32a346abcf75"
format_stamp: "Formatted at 2025-10-28 02:59:49 on dist-test-slave-97kg"
I20251028 02:59:49.639185 8338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20251028 02:59:49.641546 8353 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:49.642598 8338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:49.642706 8338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal
uuid: "d7e4cf694f71438298dd32a346abcf75"
format_stamp: "Formatted at 2025-10-28 02:59:49 on dist-test-slave-97kg"
I20251028 02:59:49.642794 8338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 02:59:49.680768 8338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 02:59:49.681480 8338 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251028 02:59:49.681674 8338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 02:59:49.682260 8338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251028 02:59:49.683359 8338 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251028 02:59:49.683413 8338 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:49.683473 8338 ts_tablet_manager.cc:616] Registered 0 tablets
I20251028 02:59:49.683502 8338 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:49.692641 8338 rpc_server.cc:307] RPC server started. Bound to: 127.7.154.131:33189
I20251028 02:59:49.692667 8466 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.154.131:33189 every 8 connection(s)
I20251028 02:59:49.693655 8338 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/data/info.pb
I20251028 02:59:49.702080 7786 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu as pid 8338
I20251028 02:59:49.702212 7786 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/instance
I20251028 02:59:49.703661 8469 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:49.703644 8468 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 02:59:49.703644 8467 heartbeater.cc:344] Connected to a master server at 127.7.154.188:44237
I20251028 02:59:49.703960 8467 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:49.703960 8468 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:49.703960 8469 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:49.704701 8467 heartbeater.cc:507] Master 127.7.154.188:44237 requested a full tablet report, sending...
I20251028 02:59:49.704691 8468 heartbeater.cc:507] Master 127.7.154.190:39091 requested a full tablet report, sending...
I20251028 02:59:49.704691 8469 heartbeater.cc:507] Master 127.7.154.189:45473 requested a full tablet report, sending...
I20251028 02:59:49.705914 7896 ts_manager.cc:194] Registered new tserver with Master: d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:49.705996 7825 ts_manager.cc:194] Registered new tserver with Master: d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:49.706470 7968 ts_manager.cc:194] Registered new tserver with Master: d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:49.706652 7896 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.7.154.131:49549
I20251028 02:59:49.720875 7786 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251028 02:59:49.735052 7786 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu with pid 7863
W20251028 02:59:49.743052 8474 connection.cc:537] client connection to 127.7.154.189:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251028 02:59:49.743665 8474 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.154.189:45473)
I20251028 02:59:49.809015 8196 heartbeater.cc:499] Master 127.7.154.189:45473 was elected leader, sending a full tablet report...
W20251028 02:59:49.809998 8196 heartbeater.cc:646] Failed to heartbeat to 127.7.154.189:45473 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111)
I20251028 02:59:49.810318 8025 catalog_manager.cc:1269] Loaded cluster ID: a891460fa9a9449bb0459a49a032f9be
I20251028 02:59:49.810390 8025 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: loading cluster ID for follower catalog manager: success
I20251028 02:59:49.812211 8025 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: acquiring CA information for follower catalog manager: success
I20251028 02:59:49.813490 8025 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251028 02:59:49.879896 8053 catalog_manager.cc:1269] Loaded cluster ID: a891460fa9a9449bb0459a49a032f9be
I20251028 02:59:49.879988 8053 catalog_manager.cc:1562] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: loading cluster ID for follower catalog manager: success
I20251028 02:59:49.881763 8053 catalog_manager.cc:1584] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: acquiring CA information for follower catalog manager: success
I20251028 02:59:49.883066 8053 catalog_manager.cc:1612] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251028 02:59:50.001163 8333 heartbeater.cc:499] Master 127.7.154.189:45473 was elected leader, sending a full tablet report...
W20251028 02:59:50.002095 8333 heartbeater.cc:646] Failed to heartbeat to 127.7.154.189:45473 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111)
I20251028 02:59:50.002909 8514 raft_consensus.cc:493] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aac264d6d2274d3283bd3251ab77e583)
I20251028 02:59:50.003018 8514 raft_consensus.cc:515] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:50.004129 8514 leader_election.cc:290] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aac264d6d2274d3283bd3251ab77e583 (127.7.154.189:45473), 9343ca8510de4eaa9945786e054a866d (127.7.154.188:44237)
I20251028 02:59:50.004441 7979 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac9bf5a198d042698c9f030a59228865" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9343ca8510de4eaa9945786e054a866d" is_pre_election: true
W20251028 02:59:50.005347 7809 leader_election.cc:336] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aac264d6d2274d3283bd3251ab77e583 (127.7.154.189:45473): Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111)
I20251028 02:59:50.005431 7809 leader_election.cc:304] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac9bf5a198d042698c9f030a59228865; no voters: 9343ca8510de4eaa9945786e054a866d, aac264d6d2274d3283bd3251ab77e583
I20251028 02:59:50.005656 8514 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251028 02:59:50.171792 8517 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aac264d6d2274d3283bd3251ab77e583)
I20251028 02:59:50.171931 8517 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:50.172399 8517 leader_election.cc:290] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ac9bf5a198d042698c9f030a59228865 (127.7.154.190:39091), aac264d6d2274d3283bd3251ab77e583 (127.7.154.189:45473)
I20251028 02:59:50.172897 7835 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9343ca8510de4eaa9945786e054a866d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ac9bf5a198d042698c9f030a59228865" is_pre_election: true
I20251028 02:59:50.173053 7835 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9343ca8510de4eaa9945786e054a866d in term 1.
W20251028 02:59:50.173113 7953 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111)
I20251028 02:59:50.173417 7956 leader_election.cc:304] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9343ca8510de4eaa9945786e054a866d, ac9bf5a198d042698c9f030a59228865; no voters:
I20251028 02:59:50.173818 8517 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251028 02:59:50.173890 8517 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 1 FOLLOWER]: Starting leader election (detected failure of leader aac264d6d2274d3283bd3251ab77e583)
I20251028 02:59:50.173947 8517 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 1 FOLLOWER]: Advancing to term 2
W20251028 02:59:50.173997 7953 leader_election.cc:336] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aac264d6d2274d3283bd3251ab77e583 (127.7.154.189:45473): Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111)
I20251028 02:59:50.179076 8517 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:50.179661 8517 leader_election.cc:290] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [CANDIDATE]: Term 2 election: Requested vote from peers ac9bf5a198d042698c9f030a59228865 (127.7.154.190:39091), aac264d6d2274d3283bd3251ab77e583 (127.7.154.189:45473)
I20251028 02:59:50.180377 7835 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9343ca8510de4eaa9945786e054a866d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ac9bf5a198d042698c9f030a59228865"
I20251028 02:59:50.180532 7835 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 1 FOLLOWER]: Advancing to term 2
W20251028 02:59:50.180759 7953 leader_election.cc:336] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aac264d6d2274d3283bd3251ab77e583 (127.7.154.189:45473): Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111)
I20251028 02:59:50.181470 7835 raft_consensus.cc:2468] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9343ca8510de4eaa9945786e054a866d in term 2.
I20251028 02:59:50.181783 7956 leader_election.cc:304] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9343ca8510de4eaa9945786e054a866d, ac9bf5a198d042698c9f030a59228865; no voters: aac264d6d2274d3283bd3251ab77e583
I20251028 02:59:50.181967 8517 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 2 FOLLOWER]: Leader election won for term 2
I20251028 02:59:50.182150 8517 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [term 2 LEADER]: Becoming Leader. State: Replica: 9343ca8510de4eaa9945786e054a866d, State: Running, Role: LEADER
I20251028 02:59:50.182467 8517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:50.183885 8520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9343ca8510de4eaa9945786e054a866d. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:50.183986 8520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: This master's current role is: LEADER
I20251028 02:59:50.184311 8522 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251028 02:59:50.185115 8522 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251028 02:59:50.185431 8522 catalog_manager.cc:1269] Loaded cluster ID: a891460fa9a9449bb0459a49a032f9be
I20251028 02:59:50.185480 8522 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251028 02:59:50.185951 8522 catalog_manager.cc:1514] Loading token signing keys...
I20251028 02:59:50.186304 8522 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Loaded TSK: 0
I20251028 02:59:50.187462 8522 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251028 02:59:50.208506 8469 heartbeater.cc:499] Master 127.7.154.189:45473 was elected leader, sending a full tablet report...
W20251028 02:59:50.209383 8469 heartbeater.cc:646] Failed to heartbeat to 127.7.154.189:45473 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111)
I20251028 02:59:50.222301 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0bdad791772c4a83a8ce20582ecb0e9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.222498 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_98df016041654d5c97171ebe5932c530"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.222560 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d0e6709eadac4e56b37c87a1ca94436b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.222244 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ba11bb13b73c4949a624d99c9407b773"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.222895 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ee857cb6a36d4a6192f2bfb82a8729b4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.223014 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d6ea48a655964c3191bfcb61d97d0c29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.222182 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1697a58fb1aa4feebc49b037d61e0b56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.222189 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bf85a28c0e2242e8b6227caba6cdf96b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.223281 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_910b250d6ad543ef83e25fff49c27625"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.223729 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee857cb6a36d4a6192f2bfb82a8729b4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.223856 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1697a58fb1aa4feebc49b037d61e0b56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.223902 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d6ea48a655964c3191bfcb61d97d0c29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.224020 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba11bb13b73c4949a624d99c9407b773 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.223902 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98df016041654d5c97171ebe5932c530 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.223729 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0e6709eadac4e56b37c87a1ca94436b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.223734 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf85a28c0e2242e8b6227caba6cdf96b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.222905 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_835c550d80864449afac41b445e6615b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.223734 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_910b250d6ad543ef83e25fff49c27625 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.224277 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0bdad791772c4a83a8ce20582ecb0e9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.224613 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_835c550d80864449afac41b445e6615b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.227906 7835 raft_consensus.cc:1275] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 2 FOLLOWER]: Refusing update from remote peer 9343ca8510de4eaa9945786e054a866d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
W20251028 02:59:50.228467 7953 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d -> Peer aac264d6d2274d3283bd3251ab77e583 (127.7.154.189:45473): Couldn't send request to peer aac264d6d2274d3283bd3251ab77e583. Status: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:50.228612 8520 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251028 02:59:50.230628 8514 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9343ca8510de4eaa9945786e054a866d. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:50.230734 8514 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:50.233880 8514 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:50.233994 8514 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:50.234175 8525 mvcc.cc:204] Tried to move back new op lower bound from 7215597118374010880 to 7215597118189735936. Current Snapshot: MvccSnapshot[applied={T|T < 7215597118374010880 or (T in {7215597118374010880})}]
I20251028 02:59:50.235718 8520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:50.235870 8520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: This master's current role is: LEADER
I20251028 02:59:50.235941 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_910b250d6ad543ef83e25fff49c27625" } new_table_name: "default.Table_01a0a26081324cd889dcfadfcff36885" modify_external_catalogs: true
I20251028 02:59:50.238476 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d6ea48a655964c3191bfcb61d97d0c29" } modify_external_catalogs: true
I20251028 02:59:50.238633 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d6ea48a655964c3191bfcb61d97d0c29" } modify_external_catalogs: true
I20251028 02:59:50.238714 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ba11bb13b73c4949a624d99c9407b773" } modify_external_catalogs: true
I20251028 02:59:50.238794 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ba11bb13b73c4949a624d99c9407b773" } modify_external_catalogs: true
I20251028 02:59:50.239073 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0bdad791772c4a83a8ce20582ecb0e9d" } modify_external_catalogs: true
I20251028 02:59:50.239154 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0bdad791772c4a83a8ce20582ecb0e9d" } modify_external_catalogs: true
I20251028 02:59:50.239676 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_98df016041654d5c97171ebe5932c530" } new_table_name: "default.Table_52b40e03037e4a22b124573e686db887" modify_external_catalogs: true
I20251028 02:59:50.240635 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_835c550d80864449afac41b445e6615b" } modify_external_catalogs: true
I20251028 02:59:50.240742 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_835c550d80864449afac41b445e6615b" } modify_external_catalogs: true
I20251028 02:59:50.242405 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d0e6709eadac4e56b37c87a1ca94436b" } modify_external_catalogs: true
I20251028 02:59:50.242470 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1697a58fb1aa4feebc49b037d61e0b56" } new_table_name: "default.Table_2204fdfdab02441eb98f08410e42c935" modify_external_catalogs: true
I20251028 02:59:50.242501 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d0e6709eadac4e56b37c87a1ca94436b" } modify_external_catalogs: true
I20251028 02:59:50.243144 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ee857cb6a36d4a6192f2bfb82a8729b4" } new_table_name: "default.Table_cb9df870fac44834a941ee972ff7ae91" modify_external_catalogs: true
W20251028 02:59:50.245786 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 211656fc4df24403be3c8f1aae01477f has no consensus state
I20251028 02:59:50.246119 7964 catalog_manager.cc:5931] Not sending DeleteTablet requests; no consensus state for tablet 80040086c85d4e8997f024746687d689
I20251028 02:59:50.247339 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bf85a28c0e2242e8b6227caba6cdf96b" } new_table_name: "default.Table_c4ca664925d14a819355309d0bd6144a" modify_external_catalogs: true
W20251028 02:59:50.247587 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9d7dacea3c474b6d848b172a0e28aa13 has no consensus state
W20251028 02:59:50.247931 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6b590e97251545979879478f7933c650 has no consensus state
W20251028 02:59:50.248287 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2386ea3c08fe48c381c7fc2321ecbc86 has no consensus state
I20251028 02:59:50.248769 7960 catalog_manager.cc:5931] Not sending DeleteTablet requests; no consensus state for tablet 34b139e4b8a540dab0e75ac25cad3734
I20251028 02:59:50.255719 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 77c521524ab64dcd9ffcc408bbc8a462 (DEFAULT_TABLE table=default.Table_0bdad791772c4a83a8ce20582ecb0e9d [id=4700b02f747a4abe9df4d6d2206cb0ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.255478 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 211656fc4df24403be3c8f1aae01477f (DEFAULT_TABLE table=default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.255478 8126 tablet_service.cc:1505] Processing CreateTablet for tablet e9578df5c0b345689701ce4070058b58 (DEFAULT_TABLE table=default.Table_835c550d80864449afac41b445e6615b [id=05cb1987d5e74910ba1d5e2edaa08762]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.256671 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77c521524ab64dcd9ffcc408bbc8a462. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.257088 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9578df5c0b345689701ce4070058b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.257265 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211656fc4df24403be3c8f1aae01477f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.259121 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 9d7dacea3c474b6d848b172a0e28aa13 (DEFAULT_TABLE table=default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.259413 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d7dacea3c474b6d848b172a0e28aa13. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.265831 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 211656fc4df24403be3c8f1aae01477f (DEFAULT_TABLE table=default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.265694 8399 tablet_service.cc:1505] Processing CreateTablet for tablet e9578df5c0b345689701ce4070058b58 (DEFAULT_TABLE table=default.Table_835c550d80864449afac41b445e6615b [id=05cb1987d5e74910ba1d5e2edaa08762]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.266180 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 9d7dacea3c474b6d848b172a0e28aa13 (DEFAULT_TABLE table=default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.265682 8401 tablet_service.cc:1505] Processing CreateTablet for tablet e76df33edf254924a75ac534ba815030 (DEFAULT_TABLE table=default.Table_d6ea48a655964c3191bfcb61d97d0c29 [id=b54776de32064719af54117d3391b3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.266824 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d7dacea3c474b6d848b172a0e28aa13. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.267164 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e76df33edf254924a75ac534ba815030. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.267347 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211656fc4df24403be3c8f1aae01477f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.267026 8264 tablet_service.cc:1505] Processing CreateTablet for tablet 77c521524ab64dcd9ffcc408bbc8a462 (DEFAULT_TABLE table=default.Table_0bdad791772c4a83a8ce20582ecb0e9d [id=4700b02f747a4abe9df4d6d2206cb0ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.267994 8264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77c521524ab64dcd9ffcc408bbc8a462. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.269529 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9578df5c0b345689701ce4070058b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.271605 8263 tablet_service.cc:1505] Processing CreateTablet for tablet e9578df5c0b345689701ce4070058b58 (DEFAULT_TABLE table=default.Table_835c550d80864449afac41b445e6615b [id=05cb1987d5e74910ba1d5e2edaa08762]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.271906 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9578df5c0b345689701ce4070058b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.273675 8545 tablet_bootstrap.cc:492] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.275542 8546 tablet_bootstrap.cc:492] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.275561 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 211656fc4df24403be3c8f1aae01477f (DEFAULT_TABLE table=default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.275861 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211656fc4df24403be3c8f1aae01477f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.275875 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 9d7dacea3c474b6d848b172a0e28aa13 (DEFAULT_TABLE table=default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.276115 8545 tablet_bootstrap.cc:654] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.276139 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d7dacea3c474b6d848b172a0e28aa13. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:50.276707 7956 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 2386ea3c08fe48c381c7fc2321ecbc86 (table default.Table_cb9df870fac44834a941ee972ff7ae91 [id=9d19f4b99b02446c8bbc1263a9b70c4e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2386ea3c08fe48c381c7fc2321ecbc86 has no consensus state
I20251028 02:59:50.277302 8545 log.cc:826] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75: Log is configured to *not* fsync() on all Append() calls
I20251028 02:59:50.278052 8546 tablet_bootstrap.cc:654] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.278366 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 77c521524ab64dcd9ffcc408bbc8a462 (DEFAULT_TABLE table=default.Table_0bdad791772c4a83a8ce20582ecb0e9d [id=4700b02f747a4abe9df4d6d2206cb0ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.278625 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77c521524ab64dcd9ffcc408bbc8a462. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.278836 8546 log.cc:826] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda: Log is configured to *not* fsync() on all Append() calls
I20251028 02:59:50.280151 8545 tablet_bootstrap.cc:492] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.280301 8545 ts_tablet_manager.cc:1403] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20251028 02:59:50.280427 8546 tablet_bootstrap.cc:492] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.280591 8546 ts_tablet_manager.cc:1403] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251028 02:59:50.280956 8265 tablet_service.cc:1505] Processing CreateTablet for tablet e76df33edf254924a75ac534ba815030 (DEFAULT_TABLE table=default.Table_d6ea48a655964c3191bfcb61d97d0c29 [id=b54776de32064719af54117d3391b3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.281215 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e76df33edf254924a75ac534ba815030. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.255483 8128 tablet_service.cc:1505] Processing CreateTablet for tablet e76df33edf254924a75ac534ba815030 (DEFAULT_TABLE table=default.Table_d6ea48a655964c3191bfcb61d97d0c29 [id=b54776de32064719af54117d3391b3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.282536 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e76df33edf254924a75ac534ba815030. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.284488 8545 raft_consensus.cc:359] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:50.284866 8545 raft_consensus.cc:385] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.285004 8545 raft_consensus.cc:740] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.285615 8545 consensus_queue.cc:260] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:50.285563 8546 raft_consensus.cc:359] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.285773 8546 raft_consensus.cc:385] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.285897 8546 raft_consensus.cc:740] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.286556 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet e9578df5c0b345689701ce4070058b58
I20251028 02:59:50.286739 8545 ts_tablet_manager.cc:1434] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20251028 02:59:50.286826 8544 tablet_bootstrap.cc:492] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.287075 8467 heartbeater.cc:499] Master 127.7.154.188:44237 was elected leader, sending a full tablet report...
W20251028 02:59:50.287554 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 533d7e48698b4b2d8fbd27c680bff299 has no consensus state
I20251028 02:59:50.287711 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet e9578df5c0b345689701ce4070058b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.287775 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet e9578df5c0b345689701ce4070058b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.288203 8264 tablet_service.cc:1552] Processing DeleteTablet for tablet e9578df5c0b345689701ce4070058b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:50.290683 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e9578df5c0b345689701ce4070058b58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9578df5c0b345689701ce4070058b58 already in progress: creating tablet
W20251028 02:59:50.290772 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet e9578df5c0b345689701ce4070058b58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9578df5c0b345689701ce4070058b58 already in progress: creating tablet
I20251028 02:59:50.291641 8546 consensus_queue.cc:260] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.292385 8544 tablet_bootstrap.cc:654] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.292910 8546 ts_tablet_manager.cc:1434] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20251028 02:59:50.293287 8544 log.cc:826] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0: Log is configured to *not* fsync() on all Append() calls
I20251028 02:59:50.293887 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 9d7dacea3c474b6d848b172a0e28aa13 (table default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]). Expected version 1 got 0
I20251028 02:59:50.294139 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 211656fc4df24403be3c8f1aae01477f (table default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]). Expected version 1 got 0
I20251028 02:59:50.294579 8546 tablet_bootstrap.cc:492] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.295022 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 2386ea3c08fe48c381c7fc2321ecbc86 (DEFAULT_TABLE table=default.Table_cb9df870fac44834a941ee972ff7ae91 [id=9d19f4b99b02446c8bbc1263a9b70c4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.295137 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 6b590e97251545979879478f7933c650 (DEFAULT_TABLE table=default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.295517 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 533d7e48698b4b2d8fbd27c680bff299 (DEFAULT_TABLE table=default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.295779 8546 tablet_bootstrap.cc:654] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.297580 8546 tablet_bootstrap.cc:492] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.297747 8546 ts_tablet_manager.cc:1403] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251028 02:59:50.298664 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2386ea3c08fe48c381c7fc2321ecbc86. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.298645 8264 tablet_service.cc:1505] Processing CreateTablet for tablet 2386ea3c08fe48c381c7fc2321ecbc86 (DEFAULT_TABLE table=default.Table_cb9df870fac44834a941ee972ff7ae91 [id=9d19f4b99b02446c8bbc1263a9b70c4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.298781 8265 tablet_service.cc:1505] Processing CreateTablet for tablet 6b590e97251545979879478f7933c650 (DEFAULT_TABLE table=default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.299014 8331 heartbeater.cc:499] Master 127.7.154.188:44237 was elected leader, sending a full tablet report...
I20251028 02:59:50.299005 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 2386ea3c08fe48c381c7fc2321ecbc86 (DEFAULT_TABLE table=default.Table_cb9df870fac44834a941ee972ff7ae91 [id=9d19f4b99b02446c8bbc1263a9b70c4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.299263 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2386ea3c08fe48c381c7fc2321ecbc86. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.299383 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet e9578df5c0b345689701ce4070058b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.300009 8546 raft_consensus.cc:359] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.300719 8264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2386ea3c08fe48c381c7fc2321ecbc86. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.301502 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 533d7e48698b4b2d8fbd27c680bff299. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.301484 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 533d7e48698b4b2d8fbd27c680bff299 (DEFAULT_TABLE table=default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.301738 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 533d7e48698b4b2d8fbd27c680bff299. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.303570 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b590e97251545979879478f7933c650. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.303838 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 6b590e97251545979879478f7933c650 (DEFAULT_TABLE table=default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.304034 8545 tablet_bootstrap.cc:492] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.304085 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b590e97251545979879478f7933c650. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.305182 8545 tablet_bootstrap.cc:654] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.306366 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 533d7e48698b4b2d8fbd27c680bff299 (DEFAULT_TABLE table=default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.306432 8545 tablet_bootstrap.cc:492] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.306496 8545 ts_tablet_manager.cc:1403] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:50.306582 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 533d7e48698b4b2d8fbd27c680bff299. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.307116 8546 raft_consensus.cc:385] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.307093 8545 raft_consensus.cc:359] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.307195 8545 raft_consensus.cc:385] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.307273 8545 raft_consensus.cc:740] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.307279 8546 raft_consensus.cc:740] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.307483 8546 consensus_queue.cc:260] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.307495 8545 consensus_queue.cc:260] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.307893 8545 ts_tablet_manager.cc:1434] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251028 02:59:50.308261 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet e76df33edf254924a75ac534ba815030
I20251028 02:59:50.308655 8546 ts_tablet_manager.cc:1434] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.011s user 0.000s sys 0.001s
I20251028 02:59:50.308851 8546 tablet_bootstrap.cc:492] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.308982 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 77c521524ab64dcd9ffcc408bbc8a462
I20251028 02:59:50.309091 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet e76df33edf254924a75ac534ba815030 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.311048 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 77c521524ab64dcd9ffcc408bbc8a462 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.311174 8546 tablet_bootstrap.cc:654] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.311779 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 77c521524ab64dcd9ffcc408bbc8a462 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:50.311975 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e76df33edf254924a75ac534ba815030 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e76df33edf254924a75ac534ba815030 already in progress: creating tablet
I20251028 02:59:50.312002 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet e76df33edf254924a75ac534ba815030 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.312527 8552 tablet_replica.cc:333] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.312915 8552 raft_consensus.cc:2243] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.312935 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 9d7dacea3c474b6d848b172a0e28aa13 (table default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]). Expected version 1 got 0
I20251028 02:59:50.313046 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 211656fc4df24403be3c8f1aae01477f (table default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]). Expected version 1 got 0
I20251028 02:59:50.313234 8552 raft_consensus.cc:2272] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.313505 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b590e97251545979879478f7933c650. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.315814 8552 ts_tablet_manager.cc:1916] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:50.316124 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 77c521524ab64dcd9ffcc408bbc8a462 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77c521524ab64dcd9ffcc408bbc8a462 already in progress: creating tablet
I20251028 02:59:50.316674 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 9d7dacea3c474b6d848b172a0e28aa13 (table default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]). Expected version 1 got 0
I20251028 02:59:50.318230 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 533d7e48698b4b2d8fbd27c680bff299 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.318228 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 77c521524ab64dcd9ffcc408bbc8a462 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.318364 7786 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
/tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.189:45473
--webserver_interface=127.7.154.189
--webserver_port=36469
--builtin_ntp_servers=127.7.154.148:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20251028 02:59:50.319190 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 9d7dacea3c474b6d848b172a0e28aa13 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.320487 8545 tablet_bootstrap.cc:492] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.321111 8552 ts_tablet_manager.cc:1929] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.321244 8552 log.cc:1199] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/e9578df5c0b345689701ce4070058b58
I20251028 02:59:50.321555 8552 ts_tablet_manager.cc:1950] T e9578df5c0b345689701ce4070058b58 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.321709 8545 tablet_bootstrap.cc:654] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.322885 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet e76df33edf254924a75ac534ba815030 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.322921 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet e9578df5c0b345689701ce4070058b58 (table default.Table_835c550d80864449afac41b445e6615b [id=05cb1987d5e74910ba1d5e2edaa08762]) successfully deleted
I20251028 02:59:50.323091 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 77c521524ab64dcd9ffcc408bbc8a462 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:50.323410 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 77c521524ab64dcd9ffcc408bbc8a462 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77c521524ab64dcd9ffcc408bbc8a462 already in progress: creating tablet
I20251028 02:59:50.323544 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 211656fc4df24403be3c8f1aae01477f from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:50.324213 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet e9578df5c0b345689701ce4070058b58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9578df5c0b345689701ce4070058b58 already in progress: creating tablet
W20251028 02:59:50.324366 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet e76df33edf254924a75ac534ba815030 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e76df33edf254924a75ac534ba815030 already in progress: creating tablet
I20251028 02:59:50.325407 8551 tablet_replica.cc:333] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.325713 8551 raft_consensus.cc:2243] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.325889 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 533d7e48698b4b2d8fbd27c680bff299
I20251028 02:59:50.325992 8551 raft_consensus.cc:2272] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.326222 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 533d7e48698b4b2d8fbd27c680bff299 deleted and replaced by tablet 85a0ee11b35545cca8db8754c0b6b7d7
I20251028 02:59:50.326998 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet e76df33edf254924a75ac534ba815030 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.327024 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet 533d7e48698b4b2d8fbd27c680bff299 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a0ee11b35545cca8db8754c0b6b7d7) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.327497 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 533d7e48698b4b2d8fbd27c680bff299 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a0ee11b35545cca8db8754c0b6b7d7) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:50.327889 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 533d7e48698b4b2d8fbd27c680bff299 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 533d7e48698b4b2d8fbd27c680bff299 already in progress: creating tablet
I20251028 02:59:50.328449 8551 ts_tablet_manager.cc:1916] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.328580 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9d7dacea3c474b6d848b172a0e28aa13
I20251028 02:59:50.328872 7964 catalog_manager.cc:6719] ReplaceTablet: tablet 9d7dacea3c474b6d848b172a0e28aa13 deleted and replaced by tablet caff8b9ed8a54ccbbcdfac2d62bc2a71
I20251028 02:59:50.329099 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 9d7dacea3c474b6d848b172a0e28aa13 with delete_type TABLET_DATA_DELETED (replaced by tablet caff8b9ed8a54ccbbcdfac2d62bc2a71) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:50.329571 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9d7dacea3c474b6d848b172a0e28aa13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9d7dacea3c474b6d848b172a0e28aa13 already in progress: creating tablet
I20251028 02:59:50.329912 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 9d7dacea3c474b6d848b172a0e28aa13 with delete_type TABLET_DATA_DELETED (replaced by tablet caff8b9ed8a54ccbbcdfac2d62bc2a71) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.330073 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet 9d7dacea3c474b6d848b172a0e28aa13 with delete_type TABLET_DATA_DELETED (replaced by tablet caff8b9ed8a54ccbbcdfac2d62bc2a71) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.330302 8552 tablet_replica.cc:333] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.330487 8552 raft_consensus.cc:2243] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.330651 8552 raft_consensus.cc:2272] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.331235 8552 ts_tablet_manager.cc:1916] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.331553 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 533d7e48698b4b2d8fbd27c680bff299 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a0ee11b35545cca8db8754c0b6b7d7) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:50.332046 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 533d7e48698b4b2d8fbd27c680bff299 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 533d7e48698b4b2d8fbd27c680bff299 already in progress: creating tablet
I20251028 02:59:50.332299 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 211656fc4df24403be3c8f1aae01477f
I20251028 02:59:50.332617 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 211656fc4df24403be3c8f1aae01477f deleted and replaced by tablet c4b5b1d7c738413f80b8aed09b597c60
I20251028 02:59:50.332809 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 211656fc4df24403be3c8f1aae01477f with delete_type TABLET_DATA_DELETED (replaced by tablet c4b5b1d7c738413f80b8aed09b597c60) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:50.333252 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 211656fc4df24403be3c8f1aae01477f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 211656fc4df24403be3c8f1aae01477f already in progress: creating tablet
I20251028 02:59:50.333392 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet 211656fc4df24403be3c8f1aae01477f with delete_type TABLET_DATA_DELETED (replaced by tablet c4b5b1d7c738413f80b8aed09b597c60) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.333551 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 211656fc4df24403be3c8f1aae01477f with delete_type TABLET_DATA_DELETED (replaced by tablet c4b5b1d7c738413f80b8aed09b597c60) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.335083 8544 tablet_bootstrap.cc:492] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.335263 8552 ts_tablet_manager.cc:1929] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.335343 8544 ts_tablet_manager.cc:1403] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.049s user 0.004s sys 0.002s
I20251028 02:59:50.335408 8552 log.cc:1199] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9d7dacea3c474b6d848b172a0e28aa13
I20251028 02:59:50.335645 8552 ts_tablet_manager.cc:1950] T 9d7dacea3c474b6d848b172a0e28aa13 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.336426 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9d7dacea3c474b6d848b172a0e28aa13 (table default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]) successfully deleted
W20251028 02:59:50.336710 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 211656fc4df24403be3c8f1aae01477f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 211656fc4df24403be3c8f1aae01477f already in progress: creating tablet
I20251028 02:59:50.337903 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 85a0ee11b35545cca8db8754c0b6b7d7 (DEFAULT_TABLE table=default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.338109 8265 tablet_service.cc:1505] Processing CreateTablet for tablet 85a0ee11b35545cca8db8754c0b6b7d7 (DEFAULT_TABLE table=default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.338223 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85a0ee11b35545cca8db8754c0b6b7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.338372 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85a0ee11b35545cca8db8754c0b6b7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.338860 8544 raft_consensus.cc:359] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.339087 8544 raft_consensus.cc:385] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.339150 8544 raft_consensus.cc:740] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.339790 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 85a0ee11b35545cca8db8754c0b6b7d7 (DEFAULT_TABLE table=default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.340073 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85a0ee11b35545cca8db8754c0b6b7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.341235 8551 ts_tablet_manager.cc:1929] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.341344 8551 log.cc:1199] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/e76df33edf254924a75ac534ba815030
I20251028 02:59:50.341652 8551 ts_tablet_manager.cc:1950] T e76df33edf254924a75ac534ba815030 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.342465 8546 tablet_bootstrap.cc:492] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.342576 8546 ts_tablet_manager.cc:1403] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.034s user 0.000s sys 0.003s
I20251028 02:59:50.342566 8544 consensus_queue.cc:260] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.343261 8546 raft_consensus.cc:359] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.343461 8546 raft_consensus.cc:385] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.343504 8546 raft_consensus.cc:740] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.343650 8546 consensus_queue.cc:260] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.343694 8544 ts_tablet_manager.cc:1434] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.008s user 0.004s sys 0.001s
I20251028 02:59:50.343891 8546 ts_tablet_manager.cc:1434] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.344036 8194 heartbeater.cc:499] Master 127.7.154.188:44237 was elected leader, sending a full tablet report...
I20251028 02:59:50.344110 8546 tablet_bootstrap.cc:492] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.344918 8544 tablet_bootstrap.cc:492] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.345186 8546 tablet_bootstrap.cc:654] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.346086 8397 tablet_service.cc:1505] Processing CreateTablet for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 (DEFAULT_TABLE table=default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.346196 8398 tablet_service.cc:1505] Processing CreateTablet for tablet c4b5b1d7c738413f80b8aed09b597c60 (DEFAULT_TABLE table=default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.346366 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.346679 8265 tablet_service.cc:1505] Processing CreateTablet for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 (DEFAULT_TABLE table=default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.346896 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.347508 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 85a0ee11b35545cca8db8754c0b6b7d7 (table default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639]). Expected version 1 got 0
I20251028 02:59:50.347630 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 2386ea3c08fe48c381c7fc2321ecbc86 (table default.Table_cb9df870fac44834a941ee972ff7ae91 [id=9d19f4b99b02446c8bbc1263a9b70c4e]). Expected version 1 got 0
I20251028 02:59:50.347652 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4b5b1d7c738413f80b8aed09b597c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.347697 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 6b590e97251545979879478f7933c650 (table default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]). Expected version 1 got 0
W20251028 02:59:50.348845 7960 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet 85a0ee11b35545cca8db8754c0b6b7d7 (table default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 85a0ee11b35545cca8db8754c0b6b7d7 consensus state has no leader
I20251028 02:59:50.348860 8263 tablet_service.cc:1505] Processing CreateTablet for tablet c4b5b1d7c738413f80b8aed09b597c60 (DEFAULT_TABLE table=default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.349098 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4b5b1d7c738413f80b8aed09b597c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.349711 8126 tablet_service.cc:1505] Processing CreateTablet for tablet c4b5b1d7c738413f80b8aed09b597c60 (DEFAULT_TABLE table=default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.349995 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4b5b1d7c738413f80b8aed09b597c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.353155 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 211656fc4df24403be3c8f1aae01477f with delete_type TABLET_DATA_DELETED (replaced by tablet c4b5b1d7c738413f80b8aed09b597c60) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.353351 8552 tablet_replica.cc:333] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.353446 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet e9578df5c0b345689701ce4070058b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.353500 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 9d7dacea3c474b6d848b172a0e28aa13 with delete_type TABLET_DATA_DELETED (replaced by tablet caff8b9ed8a54ccbbcdfac2d62bc2a71) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.353529 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 211656fc4df24403be3c8f1aae01477f with delete_type TABLET_DATA_DELETED (replaced by tablet c4b5b1d7c738413f80b8aed09b597c60) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.353484 8552 raft_consensus.cc:2243] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.353614 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 533d7e48698b4b2d8fbd27c680bff299 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a0ee11b35545cca8db8754c0b6b7d7) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.353678 8552 raft_consensus.cc:2272] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.354110 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet e76df33edf254924a75ac534ba815030 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.353446 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 77c521524ab64dcd9ffcc408bbc8a462 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.354506 8544 tablet_bootstrap.cc:654] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.354465 8124 tablet_service.cc:1505] Processing CreateTablet for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 (DEFAULT_TABLE table=default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.354543 8550 tablet_replica.cc:333] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.354712 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.354806 8550 raft_consensus.cc:2243] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.355010 8550 raft_consensus.cc:2272] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:50.355544 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e9578df5c0b345689701ce4070058b58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9578df5c0b345689701ce4070058b58 already in progress: creating tablet
W20251028 02:59:50.355686 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9d7dacea3c474b6d848b172a0e28aa13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9d7dacea3c474b6d848b172a0e28aa13 already in progress: creating tablet
W20251028 02:59:50.355772 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 211656fc4df24403be3c8f1aae01477f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 211656fc4df24403be3c8f1aae01477f already in progress: creating tablet
W20251028 02:59:50.355872 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 533d7e48698b4b2d8fbd27c680bff299 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 533d7e48698b4b2d8fbd27c680bff299 already in progress: creating tablet
W20251028 02:59:50.355973 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e76df33edf254924a75ac534ba815030 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e76df33edf254924a75ac534ba815030 already in progress: creating tablet
I20251028 02:59:50.356194 8550 ts_tablet_manager.cc:1916] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.356979 8545 tablet_bootstrap.cc:492] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.357087 8545 ts_tablet_manager.cc:1403] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.049s user 0.000s sys 0.002s
I20251028 02:59:50.357627 8552 ts_tablet_manager.cc:1916] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.357836 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet e76df33edf254924a75ac534ba815030 (table default.Table_d6ea48a655964c3191bfcb61d97d0c29 [id=b54776de32064719af54117d3391b3cd]) successfully deleted
W20251028 02:59:50.358098 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 77c521524ab64dcd9ffcc408bbc8a462 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77c521524ab64dcd9ffcc408bbc8a462 already in progress: creating tablet
I20251028 02:59:50.358345 8550 ts_tablet_manager.cc:1929] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.358440 8550 log.cc:1199] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/77c521524ab64dcd9ffcc408bbc8a462
W20251028 02:59:50.358536 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 533d7e48698b4b2d8fbd27c680bff299 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 533d7e48698b4b2d8fbd27c680bff299 already in progress: creating tablet
I20251028 02:59:50.358582 8551 tablet_replica.cc:333] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.358762 8551 raft_consensus.cc:2243] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.358865 8550 ts_tablet_manager.cc:1950] T 77c521524ab64dcd9ffcc408bbc8a462 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.358907 8551 raft_consensus.cc:2272] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.359656 8552 ts_tablet_manager.cc:1929] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.359728 8552 log.cc:1199] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/211656fc4df24403be3c8f1aae01477f
I20251028 02:59:50.359957 8552 ts_tablet_manager.cc:1950] T 211656fc4df24403be3c8f1aae01477f P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.360260 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 77c521524ab64dcd9ffcc408bbc8a462 (table default.Table_0bdad791772c4a83a8ce20582ecb0e9d [id=4700b02f747a4abe9df4d6d2206cb0ce]) successfully deleted
I20251028 02:59:50.360598 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 211656fc4df24403be3c8f1aae01477f (table default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:50.360836 8545 raft_consensus.cc:359] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.360935 8545 raft_consensus.cc:385] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.360984 8545 raft_consensus.cc:740] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.361126 8545 consensus_queue.cc:260] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.361398 8545 ts_tablet_manager.cc:1434] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251028 02:59:50.362363 8545 tablet_bootstrap.cc:492] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.363111 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet e9578df5c0b345689701ce4070058b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.363547 8545 tablet_bootstrap.cc:654] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.363590 8551 ts_tablet_manager.cc:1916] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.365132 8546 tablet_bootstrap.cc:492] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.365234 8546 ts_tablet_manager.cc:1403] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20251028 02:59:50.365744 8551 ts_tablet_manager.cc:1929] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.365839 8551 log.cc:1199] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9d7dacea3c474b6d848b172a0e28aa13
I20251028 02:59:50.365800 8546 raft_consensus.cc:359] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:50.365903 8546 raft_consensus.cc:385] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.365976 8546 raft_consensus.cc:740] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.366124 8551 ts_tablet_manager.cc:1950] T 9d7dacea3c474b6d848b172a0e28aa13 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.366150 8546 consensus_queue.cc:260] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:50.366433 8546 ts_tablet_manager.cc:1434] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251028 02:59:50.366639 8546 tablet_bootstrap.cc:492] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.368289 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet e76df33edf254924a75ac534ba815030 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.368451 8552 tablet_replica.cc:333] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.368535 8552 raft_consensus.cc:2243] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.368630 8552 raft_consensus.cc:2272] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.368800 8546 tablet_bootstrap.cc:654] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.368847 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9d7dacea3c474b6d848b172a0e28aa13 (table default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]) successfully deleted
I20251028 02:59:50.369189 8552 ts_tablet_manager.cc:1916] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:50.369310 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 211656fc4df24403be3c8f1aae01477f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 211656fc4df24403be3c8f1aae01477f already in progress: creating tablet
I20251028 02:59:50.369514 8551 tablet_replica.cc:333] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.369660 8551 raft_consensus.cc:2243] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.369846 8551 raft_consensus.cc:2272] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.370251 8551 ts_tablet_manager.cc:1916] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.370558 8546 tablet_bootstrap.cc:492] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.370705 8546 ts_tablet_manager.cc:1403] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251028 02:59:50.371214 8552 ts_tablet_manager.cc:1929] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.371287 8552 log.cc:1199] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/e76df33edf254924a75ac534ba815030
I20251028 02:59:50.371528 8552 ts_tablet_manager.cc:1950] T e76df33edf254924a75ac534ba815030 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.372063 8545 tablet_bootstrap.cc:492] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.372056 8544 tablet_bootstrap.cc:492] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.372136 8545 ts_tablet_manager.cc:1403] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:50.372140 8544 ts_tablet_manager.cc:1403] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20251028 02:59:50.372195 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet e76df33edf254924a75ac534ba815030 (table default.Table_d6ea48a655964c3191bfcb61d97d0c29 [id=b54776de32064719af54117d3391b3cd]) successfully deleted
I20251028 02:59:50.372408 8551 ts_tablet_manager.cc:1929] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.372519 8551 log.cc:1199] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/e9578df5c0b345689701ce4070058b58
I20251028 02:59:50.371529 8546 raft_consensus.cc:359] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.372615 8546 raft_consensus.cc:385] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.372606 8544 raft_consensus.cc:359] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.372664 8546 raft_consensus.cc:740] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.372601 8545 raft_consensus.cc:359] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.372758 8544 raft_consensus.cc:385] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.372789 8545 raft_consensus.cc:385] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.372795 8544 raft_consensus.cc:740] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.372833 8545 raft_consensus.cc:740] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.372846 8546 consensus_queue.cc:260] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.372923 8544 consensus_queue.cc:260] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.372953 8545 consensus_queue.cc:260] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.373157 8544 ts_tablet_manager.cc:1434] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.373215 8545 ts_tablet_manager.cc:1434] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.373395 8545 tablet_bootstrap.cc:492] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.374464 8545 tablet_bootstrap.cc:654] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.375716 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 77c521524ab64dcd9ffcc408bbc8a462 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.376724 8544 tablet_bootstrap.cc:492] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.377055 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 211656fc4df24403be3c8f1aae01477f with delete_type TABLET_DATA_DELETED (replaced by tablet c4b5b1d7c738413f80b8aed09b597c60) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:50.377072 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 77c521524ab64dcd9ffcc408bbc8a462 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77c521524ab64dcd9ffcc408bbc8a462 already in progress: creating tablet
I20251028 02:59:50.377681 8544 tablet_bootstrap.cc:654] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.378269 8550 tablet_replica.cc:333] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.378404 8550 raft_consensus.cc:2243] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.378587 8550 raft_consensus.cc:2272] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.378824 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 211656fc4df24403be3c8f1aae01477f with delete_type TABLET_DATA_DELETED (replaced by tablet c4b5b1d7c738413f80b8aed09b597c60) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.379204 8550 ts_tablet_manager.cc:1916] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.381484 8550 ts_tablet_manager.cc:1929] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.381558 8550 log.cc:1199] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/211656fc4df24403be3c8f1aae01477f
I20251028 02:59:50.381767 8550 ts_tablet_manager.cc:1950] T 211656fc4df24403be3c8f1aae01477f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.382090 8546 ts_tablet_manager.cc:1434] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.011s user 0.000s sys 0.001s
I20251028 02:59:50.382545 8551 ts_tablet_manager.cc:1950] T e9578df5c0b345689701ce4070058b58 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.382614 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 211656fc4df24403be3c8f1aae01477f (table default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:50.383282 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet e9578df5c0b345689701ce4070058b58 (table default.Table_835c550d80864449afac41b445e6615b [id=05cb1987d5e74910ba1d5e2edaa08762]) successfully deleted
I20251028 02:59:50.383527 8551 tablet_replica.cc:333] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.383651 8551 raft_consensus.cc:2243] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.383881 8551 raft_consensus.cc:2272] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.384109 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 77c521524ab64dcd9ffcc408bbc8a462 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.384531 8551 ts_tablet_manager.cc:1916] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.384593 8552 tablet_replica.cc:333] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.384706 8552 raft_consensus.cc:2243] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.384824 8552 raft_consensus.cc:2272] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.385169 8552 ts_tablet_manager.cc:1916] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.387398 8552 ts_tablet_manager.cc:1929] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.387537 8552 log.cc:1199] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/77c521524ab64dcd9ffcc408bbc8a462
I20251028 02:59:50.387986 8546 tablet_bootstrap.cc:492] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.389076 8546 tablet_bootstrap.cc:654] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.389735 8552 ts_tablet_manager.cc:1950] T 77c521524ab64dcd9ffcc408bbc8a462 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.390527 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 77c521524ab64dcd9ffcc408bbc8a462 (table default.Table_0bdad791772c4a83a8ce20582ecb0e9d [id=4700b02f747a4abe9df4d6d2206cb0ce]) successfully deleted
I20251028 02:59:50.390874 8545 tablet_bootstrap.cc:492] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.390921 8544 tablet_bootstrap.cc:492] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.390975 8545 ts_tablet_manager.cc:1403] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20251028 02:59:50.391021 8544 ts_tablet_manager.cc:1403] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20251028 02:59:50.391172 8551 ts_tablet_manager.cc:1929] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.391233 8551 log.cc:1199] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/211656fc4df24403be3c8f1aae01477f
I20251028 02:59:50.391453 8551 ts_tablet_manager.cc:1950] T 211656fc4df24403be3c8f1aae01477f P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.391584 8545 raft_consensus.cc:359] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.391585 8544 raft_consensus.cc:359] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.391681 8545 raft_consensus.cc:385] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.391687 8544 raft_consensus.cc:385] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.391712 8545 raft_consensus.cc:740] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.391716 8544 raft_consensus.cc:740] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.391836 8545 consensus_queue.cc:260] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.391845 8544 consensus_queue.cc:260] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.392078 8544 ts_tablet_manager.cc:1434] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.392108 8545 ts_tablet_manager.cc:1434] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.392274 8544 tablet_bootstrap.cc:492] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.392321 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 211656fc4df24403be3c8f1aae01477f (table default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:50.393186 8545 tablet_bootstrap.cc:492] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.393390 8544 tablet_bootstrap.cc:654] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.394172 8545 tablet_bootstrap.cc:654] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.396857 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet e9578df5c0b345689701ce4070058b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.397111 8550 tablet_replica.cc:333] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.397115 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 77c521524ab64dcd9ffcc408bbc8a462 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.397177 8546 tablet_bootstrap.cc:492] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.397280 8546 ts_tablet_manager.cc:1403] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20251028 02:59:50.397303 8550 raft_consensus.cc:2243] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.397420 8550 raft_consensus.cc:2272] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.397605 8551 tablet_replica.cc:333] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.397711 8551 raft_consensus.cc:2243] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.397815 8551 raft_consensus.cc:2272] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.397768 8546 raft_consensus.cc:359] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.397868 8546 raft_consensus.cc:385] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.397897 8546 raft_consensus.cc:740] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.398029 8546 consensus_queue.cc:260] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.398257 8546 ts_tablet_manager.cc:1434] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.398278 8550 ts_tablet_manager.cc:1916] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.398571 8545 tablet_bootstrap.cc:492] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.398643 8545 ts_tablet_manager.cc:1403] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:50.399066 8545 raft_consensus.cc:359] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.399169 8545 raft_consensus.cc:385] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.399199 8545 raft_consensus.cc:740] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.399394 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 2386ea3c08fe48c381c7fc2321ecbc86 (table default.Table_cb9df870fac44834a941ee972ff7ae91 [id=9d19f4b99b02446c8bbc1263a9b70c4e]). Expected version 1 got 0
I20251028 02:59:50.399376 8545 consensus_queue.cc:260] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.400780 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 2386ea3c08fe48c381c7fc2321ecbc86 (table default.Table_cb9df870fac44834a941ee972ff7ae91 [id=9d19f4b99b02446c8bbc1263a9b70c4e]). Expected version 1 got 0
I20251028 02:59:50.401961 8545 ts_tablet_manager.cc:1434] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:50.402508 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 2386ea3c08fe48c381c7fc2321ecbc86 (table default.Table_cb9df870fac44834a941ee972ff7ae91 [id=9d19f4b99b02446c8bbc1263a9b70c4e]). Expected version 1 got 0
I20251028 02:59:50.402607 8550 ts_tablet_manager.cc:1929] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.402683 8550 log.cc:1199] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/e9578df5c0b345689701ce4070058b58
I20251028 02:59:50.402920 8550 ts_tablet_manager.cc:1950] T e9578df5c0b345689701ce4070058b58 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.402900 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 2386ea3c08fe48c381c7fc2321ecbc86 (table default.Table_cb9df870fac44834a941ee972ff7ae91 [id=9d19f4b99b02446c8bbc1263a9b70c4e]). Expected version 1 got 0
I20251028 02:59:50.403705 8545 tablet_bootstrap.cc:492] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.403815 8544 tablet_bootstrap.cc:492] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.403945 8544 ts_tablet_manager.cc:1403] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:50.404057 8546 tablet_bootstrap.cc:492] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.404651 8544 raft_consensus.cc:359] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.404768 8544 raft_consensus.cc:385] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.404803 8544 raft_consensus.cc:740] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.404831 8545 tablet_bootstrap.cc:654] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.404984 8544 consensus_queue.cc:260] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.405050 8546 tablet_bootstrap.cc:654] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.405284 8544 ts_tablet_manager.cc:1434] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.405826 8545 tablet_bootstrap.cc:492] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.405896 8545 ts_tablet_manager.cc:1403] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:50.406327 8545 raft_consensus.cc:359] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.406428 8545 raft_consensus.cc:385] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.406458 8545 raft_consensus.cc:740] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.406571 8545 consensus_queue.cc:260] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.406788 8545 ts_tablet_manager.cc:1434] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.406968 8545 tablet_bootstrap.cc:492] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.408000 8545 tablet_bootstrap.cc:654] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.408563 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 6b590e97251545979879478f7933c650 (table default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]). Expected version 1 got 0
I20251028 02:59:50.409150 8545 tablet_bootstrap.cc:492] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.409225 8545 ts_tablet_manager.cc:1403] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:50.409667 8545 raft_consensus.cc:359] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.409802 8545 raft_consensus.cc:385] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.409842 8545 raft_consensus.cc:740] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.409957 8545 consensus_queue.cc:260] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.410215 8545 ts_tablet_manager.cc:1434] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.410298 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 6b590e97251545979879478f7933c650 (table default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]). Expected version 1 got 0
I20251028 02:59:50.410378 8545 tablet_bootstrap.cc:492] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.411415 8545 tablet_bootstrap.cc:654] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.412649 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 533d7e48698b4b2d8fbd27c680bff299 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a0ee11b35545cca8db8754c0b6b7d7) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.413957 8551 ts_tablet_manager.cc:1916] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.414495 8544 tablet_bootstrap.cc:492] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.415690 8544 tablet_bootstrap.cc:654] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.416342 8551 ts_tablet_manager.cc:1929] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.416356 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 9d7dacea3c474b6d848b172a0e28aa13 with delete_type TABLET_DATA_DELETED (replaced by tablet caff8b9ed8a54ccbbcdfac2d62bc2a71) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.416421 8551 log.cc:1199] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/77c521524ab64dcd9ffcc408bbc8a462
I20251028 02:59:50.416649 8551 ts_tablet_manager.cc:1950] T 77c521524ab64dcd9ffcc408bbc8a462 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.417188 8550 tablet_replica.cc:333] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.417246 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet e9578df5c0b345689701ce4070058b58 (table default.Table_835c550d80864449afac41b445e6615b [id=05cb1987d5e74910ba1d5e2edaa08762]) successfully deleted
I20251028 02:59:50.417351 8551 tablet_replica.cc:333] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.417376 8550 raft_consensus.cc:2243] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.417246 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 77c521524ab64dcd9ffcc408bbc8a462 (table default.Table_0bdad791772c4a83a8ce20582ecb0e9d [id=4700b02f747a4abe9df4d6d2206cb0ce]) successfully deleted
I20251028 02:59:50.417459 8551 raft_consensus.cc:2243] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.417488 8550 raft_consensus.cc:2272] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.417558 8551 raft_consensus.cc:2272] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.418001 8551 ts_tablet_manager.cc:1916] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.418017 8550 ts_tablet_manager.cc:1916] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.419950 8551 ts_tablet_manager.cc:1929] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.420071 8551 log.cc:1199] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/533d7e48698b4b2d8fbd27c680bff299
I20251028 02:59:50.420249 8550 ts_tablet_manager.cc:1929] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.420332 8551 ts_tablet_manager.cc:1950] T 533d7e48698b4b2d8fbd27c680bff299 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.420375 8550 log.cc:1199] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9d7dacea3c474b6d848b172a0e28aa13
I20251028 02:59:50.420612 8550 ts_tablet_manager.cc:1950] T 9d7dacea3c474b6d848b172a0e28aa13 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.421084 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 533d7e48698b4b2d8fbd27c680bff299 (table default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639]) successfully deleted
I20251028 02:59:50.421423 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9d7dacea3c474b6d848b172a0e28aa13 (table default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]) successfully deleted
I20251028 02:59:50.421676 8545 tablet_bootstrap.cc:492] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.421764 8545 ts_tablet_manager.cc:1403] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20251028 02:59:50.422253 8545 raft_consensus.cc:359] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.422358 8545 raft_consensus.cc:385] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.422405 8545 raft_consensus.cc:740] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.422539 8545 consensus_queue.cc:260] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.422804 8545 ts_tablet_manager.cc:1434] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.422963 8545 tablet_bootstrap.cc:492] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.423925 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 85a0ee11b35545cca8db8754c0b6b7d7 (table default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639]). Expected version 1 got 0
I20251028 02:59:50.424044 8545 tablet_bootstrap.cc:654] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.425237 8545 tablet_bootstrap.cc:492] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.425318 8545 ts_tablet_manager.cc:1403] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:50.425633 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 85a0ee11b35545cca8db8754c0b6b7d7 (table default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639]). Expected version 1 got 0
I20251028 02:59:50.425753 8545 raft_consensus.cc:359] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.425860 8545 raft_consensus.cc:385] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.425899 8545 raft_consensus.cc:740] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.426012 8545 consensus_queue.cc:260] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.426241 8545 ts_tablet_manager.cc:1434] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.426409 8545 tablet_bootstrap.cc:492] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.427533 8545 tablet_bootstrap.cc:654] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.428602 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 (table default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1]). Expected version 1 got 0
I20251028 02:59:50.428730 8545 tablet_bootstrap.cc:492] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.428817 8545 ts_tablet_manager.cc:1403] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
W20251028 02:59:50.428948 7960 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 (table default.Table_52b40e03037e4a22b124573e686db887 [id=df4041614f8a466b96efff0231711ec1], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 consensus state has no leader
I20251028 02:59:50.429275 8545 raft_consensus.cc:359] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:50.429364 8545 raft_consensus.cc:385] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.429399 8545 raft_consensus.cc:740] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.429527 8545 consensus_queue.cc:260] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:50.429827 8545 ts_tablet_manager.cc:1434] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.430123 8546 tablet_bootstrap.cc:492] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.430224 8546 ts_tablet_manager.cc:1403] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.001s
I20251028 02:59:50.430675 8546 raft_consensus.cc:359] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.430785 8546 raft_consensus.cc:385] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.430817 8546 raft_consensus.cc:740] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.430948 8546 consensus_queue.cc:260] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.431069 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet c4b5b1d7c738413f80b8aed09b597c60 (table default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a]). Expected version 1 got 0
I20251028 02:59:50.431185 8546 ts_tablet_manager.cc:1434] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251028 02:59:50.431406 8546 tablet_bootstrap.cc:492] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:50.431429 7960 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet c4b5b1d7c738413f80b8aed09b597c60 (table default.Table_01a0a26081324cd889dcfadfcff36885 [id=5790180989b54071ad08317318931f3a], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet c4b5b1d7c738413f80b8aed09b597c60 consensus state has no leader
I20251028 02:59:50.432456 8546 tablet_bootstrap.cc:654] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.435063 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a0ae61206bc24bba809cb1cd6f4b2b31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.435240 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_93bad75d10ad47e7a7fbbfee0cef6fee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.435549 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93bad75d10ad47e7a7fbbfee0cef6fee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.435552 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0ae61206bc24bba809cb1cd6f4b2b31 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.436216 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 533d7e48698b4b2d8fbd27c680bff299 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a0ee11b35545cca8db8754c0b6b7d7) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.436405 8552 tablet_replica.cc:333] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.436527 8552 raft_consensus.cc:2243] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.436637 8552 raft_consensus.cc:2272] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.436811 8544 tablet_bootstrap.cc:492] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.436906 8544 ts_tablet_manager.cc:1403] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20251028 02:59:50.437485 8544 raft_consensus.cc:359] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:50.437601 8544 raft_consensus.cc:385] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.437638 8544 raft_consensus.cc:740] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.437776 8544 consensus_queue.cc:260] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:50.438005 8552 ts_tablet_manager.cc:1916] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.438067 8544 ts_tablet_manager.cc:1434] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:50.438288 8544 tablet_bootstrap.cc:492] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.440362 8552 ts_tablet_manager.cc:1929] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.440444 8552 log.cc:1199] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/533d7e48698b4b2d8fbd27c680bff299
I20251028 02:59:50.440651 8552 ts_tablet_manager.cc:1950] T 533d7e48698b4b2d8fbd27c680bff299 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.441538 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 533d7e48698b4b2d8fbd27c680bff299 (table default.Table_c4ca664925d14a819355309d0bd6144a [id=b34ab58e43e943909c9bdd79ad69e639]) successfully deleted
I20251028 02:59:50.441886 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1f66803097e64e178e8ffeacb35d4faa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.442183 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_15bfc391c65b4379b26b45ef36d49a30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.442281 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f66803097e64e178e8ffeacb35d4faa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.442587 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15bfc391c65b4379b26b45ef36d49a30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.442728 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet e76df33edf254924a75ac534ba815030 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.443729 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d656e1f10f1e4dba925e8578af15b760"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.443888 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ddb39e36d68643c2909826b661af4128"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.443964 8544 tablet_bootstrap.cc:654] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:50.444049 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d656e1f10f1e4dba925e8578af15b760 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.444123 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ddb39e36d68643c2909826b661af4128 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.445425 8470 tablet.cc:2378] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251028 02:59:50.445694 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_9698524cb70b45048d368e203739baba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.445983 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9698524cb70b45048d368e203739baba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.446806 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c022e2c53bc7477f9709e148a559ef5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.447124 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c022e2c53bc7477f9709e148a559ef5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.447355 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_01a0a26081324cd889dcfadfcff36885" } new_table_name: "default.Table_1d20b9847b824165a7ad3d5090884817" modify_external_catalogs: true
I20251028 02:59:50.447813 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_79fe71e0a8b243c786b93a2345f7d1b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.447983 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_13e18d2c103343648ddade08c263f084"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.448112 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79fe71e0a8b243c786b93a2345f7d1b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:50.448274 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13e18d2c103343648ddade08c263f084 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.448985 8544 tablet_bootstrap.cc:492] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.449110 8544 ts_tablet_manager.cc:1403] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20251028 02:59:50.449669 8544 raft_consensus.cc:359] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.449820 8544 raft_consensus.cc:385] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.449863 8544 raft_consensus.cc:740] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.449972 8544 consensus_queue.cc:260] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.450204 8550 tablet_replica.cc:333] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.450423 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a0ae61206bc24bba809cb1cd6f4b2b31" } modify_external_catalogs: true
I20251028 02:59:50.450510 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a0ae61206bc24bba809cb1cd6f4b2b31" } modify_external_catalogs: true
I20251028 02:59:50.451121 8550 raft_consensus.cc:2243] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.452394 8124 tablet_service.cc:1505] Processing CreateTablet for tablet c3299f1ad42741659300a5bc81013c30 (DEFAULT_TABLE table=default.Table_a0ae61206bc24bba809cb1cd6f4b2b31 [id=e042458c113340aea5f5f474d3e7c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.452548 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_52b40e03037e4a22b124573e686db887" } new_table_name: "default.Table_e33fdb2a96be46e5b88dad8b0c017acc" modify_external_catalogs: true
I20251028 02:59:50.452677 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3299f1ad42741659300a5bc81013c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.452754 8398 tablet_service.cc:1505] Processing CreateTablet for tablet c3299f1ad42741659300a5bc81013c30 (DEFAULT_TABLE table=default.Table_a0ae61206bc24bba809cb1cd6f4b2b31 [id=e042458c113340aea5f5f474d3e7c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.453019 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3299f1ad42741659300a5bc81013c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.454370 8544 ts_tablet_manager.cc:1434] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20251028 02:59:50.455148 8545 tablet_bootstrap.cc:492] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.455205 8544 tablet_bootstrap.cc:492] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.455466 8263 tablet_service.cc:1505] Processing CreateTablet for tablet c3299f1ad42741659300a5bc81013c30 (DEFAULT_TABLE table=default.Table_a0ae61206bc24bba809cb1cd6f4b2b31 [id=e042458c113340aea5f5f474d3e7c61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.455747 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3299f1ad42741659300a5bc81013c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.456172 8545 tablet_bootstrap.cc:654] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.456341 8544 tablet_bootstrap.cc:654] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.459167 8546 tablet_bootstrap.cc:492] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.459257 8546 ts_tablet_manager.cc:1403] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20251028 02:59:50.459550 8550 raft_consensus.cc:2272] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.459757 8546 raft_consensus.cc:359] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.459878 8546 raft_consensus.cc:385] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.459929 8546 raft_consensus.cc:740] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.460103 8546 consensus_queue.cc:260] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.460332 8550 ts_tablet_manager.cc:1916] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.460378 8546 ts_tablet_manager.cc:1434] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.460593 8546 tablet_bootstrap.cc:492] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.461622 8546 tablet_bootstrap.cc:654] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.462992 8544 tablet_bootstrap.cc:492] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.463095 8544 ts_tablet_manager.cc:1403] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:50.463424 8545 tablet_bootstrap.cc:492] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.463523 8545 ts_tablet_manager.cc:1403] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:50.463613 8544 raft_consensus.cc:359] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.463714 8544 raft_consensus.cc:385] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.463750 8544 raft_consensus.cc:740] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.463891 8544 consensus_queue.cc:260] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.463977 8545 raft_consensus.cc:359] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.464089 8545 raft_consensus.cc:385] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.464118 8545 raft_consensus.cc:740] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.464202 8544 ts_tablet_manager.cc:1434] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.464227 8545 consensus_queue.cc:260] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.464408 8544 tablet_bootstrap.cc:492] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 02:59:50.466444 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c4b5b1d7c738413f80b8aed09b597c60 consensus state has no leader
I20251028 02:59:50.466675 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_cb9df870fac44834a941ee972ff7ae91" } new_table_name: "default.Table_5036a52b716149b2a336ce9c3a019ed1" modify_external_catalogs: true
I20251028 02:59:50.466802 8545 ts_tablet_manager.cc:1434] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:50.467083 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c4ca664925d14a819355309d0bd6144a" } new_table_name: "default.Table_3396d9a29e3148beaa9e1fadc46527d0" modify_external_catalogs: true
I20251028 02:59:50.469875 8546 tablet_bootstrap.cc:492] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.469996 8546 ts_tablet_manager.cc:1403] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 02:59:50.470592 8546 raft_consensus.cc:359] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.470719 8546 raft_consensus.cc:385] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.470764 8546 raft_consensus.cc:740] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.470901 8546 consensus_queue.cc:260] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.471203 8546 ts_tablet_manager.cc:1434] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.471415 8546 tablet_bootstrap.cc:492] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.471901 8550 ts_tablet_manager.cc:1929] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.471968 8550 log.cc:1199] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/e76df33edf254924a75ac534ba815030
I20251028 02:59:50.472174 8550 ts_tablet_manager.cc:1950] T e76df33edf254924a75ac534ba815030 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.472461 8546 tablet_bootstrap.cc:654] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.473197 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet e76df33edf254924a75ac534ba815030 (table default.Table_d6ea48a655964c3191bfcb61d97d0c29 [id=b54776de32064719af54117d3391b3cd]) successfully deleted
I20251028 02:59:50.473858 8546 tablet_bootstrap.cc:492] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.473938 8546 ts_tablet_manager.cc:1403] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:50.474068 8544 tablet_bootstrap.cc:654] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.474354 8546 raft_consensus.cc:359] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.474468 8546 raft_consensus.cc:385] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.474503 8546 raft_consensus.cc:740] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.474620 8546 consensus_queue.cc:260] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.474864 8546 ts_tablet_manager.cc:1434] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.475021 8546 tablet_bootstrap.cc:492] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.475872 8544 tablet_bootstrap.cc:492] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.475967 8544 ts_tablet_manager.cc:1403] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:50.476300 8546 tablet_bootstrap.cc:654] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.476426 8544 raft_consensus.cc:359] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.476526 8544 raft_consensus.cc:385] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.476555 8544 raft_consensus.cc:740] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.476672 8544 consensus_queue.cc:260] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.476927 8544 ts_tablet_manager.cc:1434] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.477123 8544 tablet_bootstrap.cc:492] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.478099 8544 tablet_bootstrap.cc:654] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:50.479797 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 consensus state has no leader
I20251028 02:59:50.480127 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1f66803097e64e178e8ffeacb35d4faa" } modify_external_catalogs: true
I20251028 02:59:50.480233 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1f66803097e64e178e8ffeacb35d4faa" } modify_external_catalogs: true
I20251028 02:59:50.482280 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c3299f1ad42741659300a5bc81013c30
I20251028 02:59:50.484423 8544 tablet_bootstrap.cc:492] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.484525 8544 ts_tablet_manager.cc:1403] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:50.484984 8544 raft_consensus.cc:359] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.485103 8544 raft_consensus.cc:385] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.485134 8544 raft_consensus.cc:740] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.485258 8544 consensus_queue.cc:260] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.485821 8544 ts_tablet_manager.cc:1434] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.485989 8544 tablet_bootstrap.cc:492] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.486980 8544 tablet_bootstrap.cc:654] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:50.487087 8334 tablet.cc:2378] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251028 02:59:50.488394 8544 tablet_bootstrap.cc:492] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.488483 8544 ts_tablet_manager.cc:1403] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251028 02:59:50.488619 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet c3299f1ad42741659300a5bc81013c30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.488828 8551 tablet_replica.cc:333] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.488965 8551 raft_consensus.cc:2243] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.488981 8544 raft_consensus.cc:359] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:50.489084 8551 raft_consensus.cc:2272] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.489104 8544 raft_consensus.cc:385] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.489140 8544 raft_consensus.cc:740] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.489259 8544 consensus_queue.cc:260] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:50.489542 8544 ts_tablet_manager.cc:1434] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.489537 8551 ts_tablet_manager.cc:1916] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.489714 8544 tablet_bootstrap.cc:492] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.490792 8544 tablet_bootstrap.cc:654] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:50.491191 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85a0ee11b35545cca8db8754c0b6b7d7 consensus state has no leader
I20251028 02:59:50.491951 8551 ts_tablet_manager.cc:1929] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.492026 8551 log.cc:1199] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c3299f1ad42741659300a5bc81013c30
I20251028 02:59:50.492250 8551 ts_tablet_manager.cc:1950] T c3299f1ad42741659300a5bc81013c30 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.492455 8544 tablet_bootstrap.cc:492] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.492538 8544 ts_tablet_manager.cc:1403] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251028 02:59:50.492998 8544 raft_consensus.cc:359] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.493104 8544 raft_consensus.cc:385] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.493136 8544 raft_consensus.cc:740] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.493255 8544 consensus_queue.cc:260] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.493516 8544 ts_tablet_manager.cc:1434] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.493687 8544 tablet_bootstrap.cc:492] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.494431 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet c3299f1ad42741659300a5bc81013c30 (table default.Table_a0ae61206bc24bba809cb1cd6f4b2b31 [id=e042458c113340aea5f5f474d3e7c61b]) successfully deleted
I20251028 02:59:50.494671 8544 tablet_bootstrap.cc:654] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.494863 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_9698524cb70b45048d368e203739baba" } modify_external_catalogs: true
I20251028 02:59:50.494968 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_9698524cb70b45048d368e203739baba" } modify_external_catalogs: true
I20251028 02:59:50.495033 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_93bad75d10ad47e7a7fbbfee0cef6fee" } new_table_name: "default.Table_78628adeb83d46c1913ffa57d5336dcc" modify_external_catalogs: true
I20251028 02:59:50.498787 8123 tablet_service.cc:1505] Processing CreateTablet for tablet bd3398b5e77d494bb821a2daf73fe593 (DEFAULT_TABLE table=default.Table_1f66803097e64e178e8ffeacb35d4faa [id=0baa5ac1f0d241d29fdb43f98433b9b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.499130 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3398b5e77d494bb821a2daf73fe593. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.499364 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 3df041f16a0448839351bbc768633b69 (DEFAULT_TABLE table=default.Table_93bad75d10ad47e7a7fbbfee0cef6fee [id=a277ab5f37b54d53816ab230cf0690a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.499590 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3df041f16a0448839351bbc768633b69. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:50.502153 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3df041f16a0448839351bbc768633b69 consensus state has no leader
I20251028 02:59:50.502398 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet bd3398b5e77d494bb821a2daf73fe593
I20251028 02:59:50.502782 7960 catalog_manager.cc:5931] Not sending DeleteTablet requests; no consensus state for tablet 484bcfaa79af4d83981fc60a8711f499
I20251028 02:59:50.502904 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet bd3398b5e77d494bb821a2daf73fe593 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.503059 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 6b590e97251545979879478f7933c650 (table default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]). Expected version 1 got 0
I20251028 02:59:50.503233 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet c3299f1ad42741659300a5bc81013c30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.503475 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet c3299f1ad42741659300a5bc81013c30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:50.503611 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet bd3398b5e77d494bb821a2daf73fe593 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd3398b5e77d494bb821a2daf73fe593 already in progress: creating tablet
W20251028 02:59:50.504175 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet c3299f1ad42741659300a5bc81013c30 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c3299f1ad42741659300a5bc81013c30 already in progress: creating tablet
W20251028 02:59:50.507655 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet c3299f1ad42741659300a5bc81013c30 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c3299f1ad42741659300a5bc81013c30 already in progress: creating tablet
I20251028 02:59:50.508353 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 3df041f16a0448839351bbc768633b69 (DEFAULT_TABLE table=default.Table_93bad75d10ad47e7a7fbbfee0cef6fee [id=a277ab5f37b54d53816ab230cf0690a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.508666 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3df041f16a0448839351bbc768633b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.510953 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet c3299f1ad42741659300a5bc81013c30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.511157 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet bd3398b5e77d494bb821a2daf73fe593 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.511349 8545 tablet_bootstrap.cc:492] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.512555 8545 tablet_bootstrap.cc:654] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.513640 8398 tablet_service.cc:1505] Processing CreateTablet for tablet bd3398b5e77d494bb821a2daf73fe593 (DEFAULT_TABLE table=default.Table_1f66803097e64e178e8ffeacb35d4faa [id=0baa5ac1f0d241d29fdb43f98433b9b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.513850 8545 tablet_bootstrap.cc:492] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.513907 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3398b5e77d494bb821a2daf73fe593. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.513942 8545 ts_tablet_manager.cc:1403] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:50.514505 8545 raft_consensus.cc:359] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.514627 8545 raft_consensus.cc:385] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.514664 8545 raft_consensus.cc:740] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.514784 8545 consensus_queue.cc:260] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.516706 8264 tablet_service.cc:1552] Processing DeleteTablet for tablet bd3398b5e77d494bb821a2daf73fe593 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.516863 8265 tablet_service.cc:1505] Processing CreateTablet for tablet 3df041f16a0448839351bbc768633b69 (DEFAULT_TABLE table=default.Table_93bad75d10ad47e7a7fbbfee0cef6fee [id=a277ab5f37b54d53816ab230cf0690a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.517167 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3df041f16a0448839351bbc768633b69. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:50.517680 7955 catalog_manager.cc:4952] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet bd3398b5e77d494bb821a2daf73fe593 because the tablet was not found. No further retry: Not found: Tablet not found: bd3398b5e77d494bb821a2daf73fe593
W20251028 02:59:50.518115 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet c3299f1ad42741659300a5bc81013c30 because the tablet was not found. No further retry: Not found: Tablet not found: c3299f1ad42741659300a5bc81013c30
I20251028 02:59:50.518121 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ddb39e36d68643c2909826b661af4128" } modify_external_catalogs: true
W20251028 02:59:50.518252 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet bd3398b5e77d494bb821a2daf73fe593 because the tablet was not found. No further retry: Not found: Tablet not found: bd3398b5e77d494bb821a2daf73fe593
I20251028 02:59:50.518291 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ddb39e36d68643c2909826b661af4128" } modify_external_catalogs: true
I20251028 02:59:50.518528 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_13e18d2c103343648ddade08c263f084" } modify_external_catalogs: true
I20251028 02:59:50.518625 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_13e18d2c103343648ddade08c263f084" } modify_external_catalogs: true
I20251028 02:59:50.518658 8545 ts_tablet_manager.cc:1434] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:50.519066 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 2386ea3c08fe48c381c7fc2321ecbc86 (table default.Table_cb9df870fac44834a941ee972ff7ae91 [id=9d19f4b99b02446c8bbc1263a9b70c4e]). Expected version 1 got 0
W20251028 02:59:50.519361 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2386ea3c08fe48c381c7fc2321ecbc86 consensus state has no leader
I20251028 02:59:50.519995 8263 tablet_service.cc:1505] Processing CreateTablet for tablet bd3398b5e77d494bb821a2daf73fe593 (DEFAULT_TABLE table=default.Table_1f66803097e64e178e8ffeacb35d4faa [id=0baa5ac1f0d241d29fdb43f98433b9b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.520203 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]). Expected version 1 got 0
I20251028 02:59:50.520277 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3398b5e77d494bb821a2daf73fe593. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.521811 7969 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]). Expected version 1 got 0
I20251028 02:59:50.521858 8545 tablet_bootstrap.cc:492] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.524251 8263 tablet_service.cc:1505] Processing CreateTablet for tablet d09229ca82eb49f2b4214b238c3ec097 (DEFAULT_TABLE table=default.Table_c022e2c53bc7477f9709e148a559ef5b [id=7fe387bad5e740429e327d42f1efde22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.524617 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d09229ca82eb49f2b4214b238c3ec097. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.526412 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 6b590e97251545979879478f7933c650 (table default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]). Expected version 1 got 0
I20251028 02:59:50.526532 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 85a0ee11b35545cca8db8754c0b6b7d7 (table default.Table_3396d9a29e3148beaa9e1fadc46527d0 [id=b34ab58e43e943909c9bdd79ad69e639]). Expected version 2 got 0
I20251028 02:59:50.526613 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 (table default.Table_e33fdb2a96be46e5b88dad8b0c017acc [id=df4041614f8a466b96efff0231711ec1]). Expected version 2 got 0
I20251028 02:59:50.526686 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet c4b5b1d7c738413f80b8aed09b597c60 (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:50.527029 8126 tablet_service.cc:1505] Processing CreateTablet for tablet f74e7c0e7d634679bb70cf3515aafe22 (DEFAULT_TABLE table=default.Table_13e18d2c103343648ddade08c263f084 [id=93f5552c487f4ad49d4a1cdf54ea08e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.527259 8398 tablet_service.cc:1505] Processing CreateTablet for tablet d09229ca82eb49f2b4214b238c3ec097 (DEFAULT_TABLE table=default.Table_c022e2c53bc7477f9709e148a559ef5b [id=7fe387bad5e740429e327d42f1efde22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.527349 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f74e7c0e7d634679bb70cf3515aafe22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.527705 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d09229ca82eb49f2b4214b238c3ec097. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.527936 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 533d7e48698b4b2d8fbd27c680bff299 with delete_type TABLET_DATA_DELETED (replaced by tablet 85a0ee11b35545cca8db8754c0b6b7d7) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.528127 8550 tablet_replica.cc:333] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.528239 8550 raft_consensus.cc:2243] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.528345 8550 raft_consensus.cc:2272] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.528741 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 85a0ee11b35545cca8db8754c0b6b7d7 (table default.Table_3396d9a29e3148beaa9e1fadc46527d0 [id=b34ab58e43e943909c9bdd79ad69e639]). Expected version 2 got 0
I20251028 02:59:50.528856 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 (table default.Table_e33fdb2a96be46e5b88dad8b0c017acc [id=df4041614f8a466b96efff0231711ec1]). Expected version 2 got 0
I20251028 02:59:50.529645 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 (DEFAULT_TABLE table=default.Table_15bfc391c65b4379b26b45ef36d49a30 [id=a9662d8045574f0abf81c9e6bc2f180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.529850 8550 ts_tablet_manager.cc:1916] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.529898 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.530179 8401 tablet_service.cc:1505] Processing CreateTablet for tablet f74e7c0e7d634679bb70cf3515aafe22 (DEFAULT_TABLE table=default.Table_13e18d2c103343648ddade08c263f084 [id=93f5552c487f4ad49d4a1cdf54ea08e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.530404 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f74e7c0e7d634679bb70cf3515aafe22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.532048 8550 ts_tablet_manager.cc:1929] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.532121 8550 log.cc:1199] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/533d7e48698b4b2d8fbd27c680bff299
I20251028 02:59:50.532081 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 206b71b0f10c42fd81f291a27fffc8a2 (DEFAULT_TABLE table=default.Table_79fe71e0a8b243c786b93a2345f7d1b8 [id=d0c0560068b54a5e98d13b2196c75aa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.532294 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 206b71b0f10c42fd81f291a27fffc8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.532332 8550 ts_tablet_manager.cc:1950] T 533d7e48698b4b2d8fbd27c680bff299 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.533356 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 206b71b0f10c42fd81f291a27fffc8a2 (DEFAULT_TABLE table=default.Table_79fe71e0a8b243c786b93a2345f7d1b8 [id=d0c0560068b54a5e98d13b2196c75aa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.533583 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 206b71b0f10c42fd81f291a27fffc8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.535269 8544 tablet_bootstrap.cc:492] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.535385 8544 ts_tablet_manager.cc:1403] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.002s
I20251028 02:59:50.535951 8544 raft_consensus.cc:359] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.536082 8544 raft_consensus.cc:385] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.536124 8544 raft_consensus.cc:740] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.536245 8544 consensus_queue.cc:260] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.536541 8544 ts_tablet_manager.cc:1434] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.536536 8397 tablet_service.cc:1505] Processing CreateTablet for tablet a3d108be7a734497a9619cd7c097b2e5 (DEFAULT_TABLE table=default.Table_ddb39e36d68643c2909826b661af4128 [id=8f89769a43034f7d9f783602532e7fef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.536744 8544 tablet_bootstrap.cc:492] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.536780 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d108be7a734497a9619cd7c097b2e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.537729 8544 tablet_bootstrap.cc:654] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.538348 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet bd3398b5e77d494bb821a2daf73fe593 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.539086 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 29396aa23c734dc78983623aca442ba6 (DEFAULT_TABLE table=default.Table_d656e1f10f1e4dba925e8578af15b760 [id=85b3a3370fda46dcb686d811f1b12172]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.539363 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29396aa23c734dc78983623aca442ba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.539496 8264 tablet_service.cc:1505] Processing CreateTablet for tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 (DEFAULT_TABLE table=default.Table_15bfc391c65b4379b26b45ef36d49a30 [id=a9662d8045574f0abf81c9e6bc2f180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.539574 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 533d7e48698b4b2d8fbd27c680bff299 (table default.Table_3396d9a29e3148beaa9e1fadc46527d0 [id=b34ab58e43e943909c9bdd79ad69e639]) successfully deleted
I20251028 02:59:50.539742 8264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.541312 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 (DEFAULT_TABLE table=default.Table_15bfc391c65b4379b26b45ef36d49a30 [id=a9662d8045574f0abf81c9e6bc2f180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.541538 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.542001 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 206b71b0f10c42fd81f291a27fffc8a2 (DEFAULT_TABLE table=default.Table_79fe71e0a8b243c786b93a2345f7d1b8 [id=d0c0560068b54a5e98d13b2196c75aa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.542228 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 206b71b0f10c42fd81f291a27fffc8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.543570 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 29396aa23c734dc78983623aca442ba6 (DEFAULT_TABLE table=default.Table_d656e1f10f1e4dba925e8578af15b760 [id=85b3a3370fda46dcb686d811f1b12172]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:50.543605 8197 tablet.cc:2378] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251028 02:59:50.543800 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29396aa23c734dc78983623aca442ba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.544416 8124 tablet_service.cc:1505] Processing CreateTablet for tablet d09229ca82eb49f2b4214b238c3ec097 (DEFAULT_TABLE table=default.Table_c022e2c53bc7477f9709e148a559ef5b [id=7fe387bad5e740429e327d42f1efde22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.544678 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d09229ca82eb49f2b4214b238c3ec097. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.545670 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 6b590e97251545979879478f7933c650 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.545867 8260 tablet_service.cc:1505] Processing CreateTablet for tablet a3d108be7a734497a9619cd7c097b2e5 (DEFAULT_TABLE table=default.Table_ddb39e36d68643c2909826b661af4128 [id=8f89769a43034f7d9f783602532e7fef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.546115 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d108be7a734497a9619cd7c097b2e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.546820 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 6b590e97251545979879478f7933c650 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.548624 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 6b590e97251545979879478f7933c650 from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:50.549222 8559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20251028 02:59:50.549564 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet c3299f1ad42741659300a5bc81013c30 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.549698 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 29396aa23c734dc78983623aca442ba6 (DEFAULT_TABLE table=default.Table_d656e1f10f1e4dba925e8578af15b760 [id=85b3a3370fda46dcb686d811f1b12172]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.549974 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29396aa23c734dc78983623aca442ba6. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:50.549978 8559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251028 02:59:50.550042 8559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251028 02:59:50.551412 8265 tablet_service.cc:1505] Processing CreateTablet for tablet f74e7c0e7d634679bb70cf3515aafe22 (DEFAULT_TABLE table=default.Table_13e18d2c103343648ddade08c263f084 [id=93f5552c487f4ad49d4a1cdf54ea08e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.551674 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f74e7c0e7d634679bb70cf3515aafe22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.552695 8123 tablet_service.cc:1505] Processing CreateTablet for tablet a3d108be7a734497a9619cd7c097b2e5 (DEFAULT_TABLE table=default.Table_ddb39e36d68643c2909826b661af4128 [id=8f89769a43034f7d9f783602532e7fef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.552922 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d108be7a734497a9619cd7c097b2e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.553578 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a3d108be7a734497a9619cd7c097b2e5
I20251028 02:59:50.554972 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet c3299f1ad42741659300a5bc81013c30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.554991 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet a3d108be7a734497a9619cd7c097b2e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.555143 8550 tablet_replica.cc:333] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
W20251028 02:59:50.555191 8559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251028 02:59:50.555238 8559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251028 02:59:50.555267 8559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20251028 02:59:50.555260 8550 raft_consensus.cc:2243] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:50.555378 8559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251028 02:59:50.555404 8550 raft_consensus.cc:2272] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.555608 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet a3d108be7a734497a9619cd7c097b2e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.556047 8550 ts_tablet_manager.cc:1916] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:50.556100 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a3d108be7a734497a9619cd7c097b2e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3d108be7a734497a9619cd7c097b2e5 already in progress: creating tablet
W20251028 02:59:50.556226 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a3d108be7a734497a9619cd7c097b2e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3d108be7a734497a9619cd7c097b2e5 already in progress: creating tablet
I20251028 02:59:50.557907 8550 ts_tablet_manager.cc:1929] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.557965 8550 log.cc:1199] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c3299f1ad42741659300a5bc81013c30
I20251028 02:59:50.558166 8550 ts_tablet_manager.cc:1950] T c3299f1ad42741659300a5bc81013c30 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.559051 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet c3299f1ad42741659300a5bc81013c30 (table default.Table_a0ae61206bc24bba809cb1cd6f4b2b31 [id=e042458c113340aea5f5f474d3e7c61b]) successfully deleted
I20251028 02:59:50.559667 8546 tablet_bootstrap.cc:492] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.559823 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet a3d108be7a734497a9619cd7c097b2e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.559993 8546 ts_tablet_manager.cc:1403] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.085s user 0.002s sys 0.000s
W20251028 02:59:50.560426 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a3d108be7a734497a9619cd7c097b2e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3d108be7a734497a9619cd7c097b2e5 already in progress: creating tablet
I20251028 02:59:50.561124 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f74e7c0e7d634679bb70cf3515aafe22
I20251028 02:59:50.561275 8559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.154.148:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.189:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.154.189
--webserver_port=36469
--never_fsync=true
--heap_profile_path=/tmp/kudu.8559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 60f5e5267b92c39485a66121d3ce3cc7ef57b0e0
build type DEBUG
built by None at 28 Oct 2025 02:43:12 UTC on 24a791456cd2
build id 8750
I20251028 02:59:50.561471 8545 tablet_bootstrap.cc:654] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.562124 8546 raft_consensus.cc:359] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:50.562232 8546 raft_consensus.cc:385] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.562261 8546 raft_consensus.cc:740] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.562378 8546 consensus_queue.cc:260] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:50.562780 8559 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251028 02:59:50.564340 8559 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251028 02:59:50.565665 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet f74e7c0e7d634679bb70cf3515aafe22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.565836 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet c4b5b1d7c738413f80b8aed09b597c60 (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:50.566738 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet f74e7c0e7d634679bb70cf3515aafe22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.566932 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet f74e7c0e7d634679bb70cf3515aafe22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.567239 8546 ts_tablet_manager.cc:1434] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20251028 02:59:50.567516 8546 tablet_bootstrap.cc:492] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.567411 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6b590e97251545979879478f7933c650
I20251028 02:59:50.568612 8546 tablet_bootstrap.cc:654] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.568704 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 6b590e97251545979879478f7933c650 deleted and replaced by tablet 5444c480b4c84863bfb85f6367227005
I20251028 02:59:50.570039 8544 tablet_bootstrap.cc:492] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.570142 8544 ts_tablet_manager.cc:1403] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
W20251028 02:59:50.568398 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f74e7c0e7d634679bb70cf3515aafe22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f74e7c0e7d634679bb70cf3515aafe22 already in progress: creating tablet
I20251028 02:59:50.570637 8544 raft_consensus.cc:359] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.570745 8544 raft_consensus.cc:385] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.570781 8544 raft_consensus.cc:740] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.570791 8545 tablet_bootstrap.cc:492] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.570884 8545 ts_tablet_manager.cc:1403] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.049s user 0.002s sys 0.000s
I20251028 02:59:50.570909 8544 consensus_queue.cc:260] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.571089 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 6b590e97251545979879478f7933c650 with delete_type TABLET_DATA_DELETED (replaced by tablet 5444c480b4c84863bfb85f6367227005) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.571213 8544 ts_tablet_manager.cc:1434] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.571424 8544 tablet_bootstrap.cc:492] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.572373 8550 tablet_replica.cc:333] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
W20251028 02:59:50.572345 8602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:50.572444 8544 tablet_bootstrap.cc:654] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.572547 8550 raft_consensus.cc:2243] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:50.566172 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f74e7c0e7d634679bb70cf3515aafe22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f74e7c0e7d634679bb70cf3515aafe22 already in progress: creating tablet
W20251028 02:59:50.572949 8600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:50.573113 8550 raft_consensus.cc:2272] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:50.573235 8599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:50.573972 8550 ts_tablet_manager.cc:1916] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.574035 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 6b590e97251545979879478f7933c650 with delete_type TABLET_DATA_DELETED (replaced by tablet 5444c480b4c84863bfb85f6367227005) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.574206 8551 tablet_replica.cc:333] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.574337 8551 raft_consensus.cc:2243] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.574429 8551 raft_consensus.cc:2272] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.574465 8545 raft_consensus.cc:359] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.574575 8545 raft_consensus.cc:385] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.574609 8545 raft_consensus.cc:740] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.574752 8545 consensus_queue.cc:260] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.574939 8551 ts_tablet_manager.cc:1916] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.575100 8545 ts_tablet_manager.cc:1434] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.575258 8545 tablet_bootstrap.cc:492] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.576349 8545 tablet_bootstrap.cc:654] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.577144 8551 ts_tablet_manager.cc:1929] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.577205 8551 log.cc:1199] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/6b590e97251545979879478f7933c650
I20251028 02:59:50.577414 8551 ts_tablet_manager.cc:1950] T 6b590e97251545979879478f7933c650 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.577654 8550 ts_tablet_manager.cc:1929] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.577715 8550 log.cc:1199] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/6b590e97251545979879478f7933c650
I20251028 02:59:50.577903 8550 ts_tablet_manager.cc:1950] T 6b590e97251545979879478f7933c650 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.578123 8559 server_base.cc:1047] running on GCE node
I20251028 02:59:50.578866 8559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251028 02:59:50.579247 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet 6b590e97251545979879478f7933c650 with delete_type TABLET_DATA_DELETED (replaced by tablet 5444c480b4c84863bfb85f6367227005) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.579447 8552 tablet_replica.cc:333] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 02:59:50.579514 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet f74e7c0e7d634679bb70cf3515aafe22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f74e7c0e7d634679bb70cf3515aafe22 already in progress: creating tablet
I20251028 02:59:50.579571 8552 raft_consensus.cc:2243] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.579674 8552 raft_consensus.cc:2272] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.579739 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 6b590e97251545979879478f7933c650 (table default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]) successfully deleted
I20251028 02:59:50.580200 8552 ts_tablet_manager.cc:1916] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.580210 8559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251028 02:59:50.580251 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet bd3398b5e77d494bb821a2daf73fe593 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.580772 8545 tablet_bootstrap.cc:492] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.577898 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet bd3398b5e77d494bb821a2daf73fe593 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.580904 8545 ts_tablet_manager.cc:1403] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:50.581233 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 6b590e97251545979879478f7933c650 (table default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]) successfully deleted
I20251028 02:59:50.581509 8550 tablet_replica.cc:333] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.581485 8545 raft_consensus.cc:359] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.581645 8545 raft_consensus.cc:385] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.581676 8550 raft_consensus.cc:2243] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.581718 8545 raft_consensus.cc:740] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.581794 8550 raft_consensus.cc:2272] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.581979 8545 consensus_queue.cc:260] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.582358 8545 ts_tablet_manager.cc:1434] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.582573 8545 tablet_bootstrap.cc:492] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.582593 8552 ts_tablet_manager.cc:1929] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.582671 8552 log.cc:1199] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/6b590e97251545979879478f7933c650
I20251028 02:59:50.582886 8552 ts_tablet_manager.cc:1950] T 6b590e97251545979879478f7933c650 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.581781 8551 tablet_replica.cc:333] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.583551 8551 raft_consensus.cc:2243] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.583653 8551 raft_consensus.cc:2272] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.584139 8551 ts_tablet_manager.cc:1916] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.584522 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet c4b5b1d7c738413f80b8aed09b597c60 (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:50.585328 8265 tablet_service.cc:1505] Processing CreateTablet for tablet 5444c480b4c84863bfb85f6367227005 (DEFAULT_TABLE table=default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.585610 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5444c480b4c84863bfb85f6367227005. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.586309 8551 ts_tablet_manager.cc:1929] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.586382 8551 log.cc:1199] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/bd3398b5e77d494bb821a2daf73fe593
I20251028 02:59:50.586439 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 5444c480b4c84863bfb85f6367227005 (DEFAULT_TABLE table=default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.586576 8551 ts_tablet_manager.cc:1950] T bd3398b5e77d494bb821a2daf73fe593 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.586722 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5444c480b4c84863bfb85f6367227005. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.587802 8545 tablet_bootstrap.cc:654] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.588740 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet bd3398b5e77d494bb821a2daf73fe593 (table default.Table_1f66803097e64e178e8ffeacb35d4faa [id=0baa5ac1f0d241d29fdb43f98433b9b7]) successfully deleted
I20251028 02:59:50.588788 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet f74e7c0e7d634679bb70cf3515aafe22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.588943 8551 tablet_replica.cc:333] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.589069 8551 raft_consensus.cc:2243] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.589085 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 5444c480b4c84863bfb85f6367227005 (DEFAULT_TABLE table=default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.589185 8551 raft_consensus.cc:2272] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.589339 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5444c480b4c84863bfb85f6367227005. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.589412 8550 ts_tablet_manager.cc:1916] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.589672 8551 ts_tablet_manager.cc:1916] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.591209 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 6b590e97251545979879478f7933c650 (table default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]) successfully deleted
I20251028 02:59:50.591403 8550 ts_tablet_manager.cc:1929] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.591490 8550 log.cc:1199] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/bd3398b5e77d494bb821a2daf73fe593
I20251028 02:59:50.591580 8551 ts_tablet_manager.cc:1929] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.591667 8551 log.cc:1199] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/f74e7c0e7d634679bb70cf3515aafe22
I20251028 02:59:50.591702 8546 tablet_bootstrap.cc:492] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.591820 8546 ts_tablet_manager.cc:1403] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20251028 02:59:50.591827 8550 ts_tablet_manager.cc:1950] T bd3398b5e77d494bb821a2daf73fe593 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.591873 8551 ts_tablet_manager.cc:1950] T f74e7c0e7d634679bb70cf3515aafe22 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.592368 8546 raft_consensus.cc:359] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.592499 8546 raft_consensus.cc:385] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.592521 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet f74e7c0e7d634679bb70cf3515aafe22 (table default.Table_13e18d2c103343648ddade08c263f084 [id=93f5552c487f4ad49d4a1cdf54ea08e7]) successfully deleted
I20251028 02:59:50.592550 8546 raft_consensus.cc:740] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.592684 8546 consensus_queue.cc:260] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.592938 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet bd3398b5e77d494bb821a2daf73fe593 (table default.Table_1f66803097e64e178e8ffeacb35d4faa [id=0baa5ac1f0d241d29fdb43f98433b9b7]) successfully deleted
I20251028 02:59:50.593003 8545 tablet_bootstrap.cc:492] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.593022 8546 ts_tablet_manager.cc:1434] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.593091 8545 ts_tablet_manager.cc:1403] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20251028 02:59:50.593209 8546 tablet_bootstrap.cc:492] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.593528 8545 raft_consensus.cc:359] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.593628 8545 raft_consensus.cc:385] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.593660 8545 raft_consensus.cc:740] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.593782 8545 consensus_queue.cc:260] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.594079 8545 ts_tablet_manager.cc:1434] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.594286 8545 tablet_bootstrap.cc:492] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.594305 8546 tablet_bootstrap.cc:654] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.595275 8545 tablet_bootstrap.cc:654] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.595568 8544 tablet_bootstrap.cc:492] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.595665 8544 ts_tablet_manager.cc:1403] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20251028 02:59:50.596174 8544 raft_consensus.cc:359] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.596297 8544 raft_consensus.cc:385] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.596343 8544 raft_consensus.cc:740] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.596468 8544 consensus_queue.cc:260] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.596733 8544 ts_tablet_manager.cc:1434] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.596943 8544 tablet_bootstrap.cc:492] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.597101 8545 tablet_bootstrap.cc:492] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.597174 8545 ts_tablet_manager.cc:1403] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:50.597604 8545 raft_consensus.cc:359] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.597711 8545 raft_consensus.cc:385] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.597744 8545 raft_consensus.cc:740] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.597855 8545 consensus_queue.cc:260] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.597961 8544 tablet_bootstrap.cc:654] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.598140 8545 ts_tablet_manager.cc:1434] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.598316 8545 tablet_bootstrap.cc:492] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.599328 8545 tablet_bootstrap.cc:654] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.600404 8545 tablet_bootstrap.cc:492] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.600409 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet c3299f1ad42741659300a5bc81013c30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.600484 8545 ts_tablet_manager.cc:1403] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20251028 02:59:50.600574 8552 tablet_replica.cc:333] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.600694 8552 raft_consensus.cc:2243] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.600737 8546 tablet_bootstrap.cc:492] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.600804 8552 raft_consensus.cc:2272] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.600834 8546 ts_tablet_manager.cc:1403] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20251028 02:59:50.600907 8545 raft_consensus.cc:359] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.600999 8545 raft_consensus.cc:385] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.601032 8545 raft_consensus.cc:740] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.601173 8545 consensus_queue.cc:260] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.601338 8546 raft_consensus.cc:359] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.601421 8552 ts_tablet_manager.cc:1916] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.601447 8545 ts_tablet_manager.cc:1434] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.601439 8546 raft_consensus.cc:385] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.601485 8546 raft_consensus.cc:740] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.601614 8545 tablet_bootstrap.cc:492] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.601608 8546 consensus_queue.cc:260] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.601891 8546 ts_tablet_manager.cc:1434] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.602067 8546 tablet_bootstrap.cc:492] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.602684 8545 tablet_bootstrap.cc:654] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.603053 8546 tablet_bootstrap.cc:654] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.603106 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]). Expected version 1 got 0
I20251028 02:59:50.603752 8552 ts_tablet_manager.cc:1929] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.603827 8552 log.cc:1199] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c3299f1ad42741659300a5bc81013c30
I20251028 02:59:50.604038 8552 ts_tablet_manager.cc:1950] T c3299f1ad42741659300a5bc81013c30 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.604271 8544 tablet_bootstrap.cc:492] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.604362 8544 ts_tablet_manager.cc:1403] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:50.604532 8545 tablet_bootstrap.cc:492] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.604604 8545 ts_tablet_manager.cc:1403] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20251028 02:59:50.604882 8544 raft_consensus.cc:359] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.604979 8544 raft_consensus.cc:385] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.605010 8544 raft_consensus.cc:740] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.605098 8545 raft_consensus.cc:359] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.605197 8545 raft_consensus.cc:385] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.605132 8544 consensus_queue.cc:260] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.605230 8545 raft_consensus.cc:740] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.605331 8545 consensus_queue.cc:260] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.605427 8544 ts_tablet_manager.cc:1434] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.605576 8545 ts_tablet_manager.cc:1434] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.605619 8544 tablet_bootstrap.cc:492] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.605739 8545 tablet_bootstrap.cc:492] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.605906 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]). Expected version 1 got 0
I20251028 02:59:50.606702 8544 tablet_bootstrap.cc:654] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.606714 8545 tablet_bootstrap.cc:654] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.607929 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c3299f1ad42741659300a5bc81013c30 (table default.Table_a0ae61206bc24bba809cb1cd6f4b2b31 [id=e042458c113340aea5f5f474d3e7c61b]) successfully deleted
I20251028 02:59:50.608145 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet a3d108be7a734497a9619cd7c097b2e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.608304 8551 tablet_replica.cc:333] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.608436 8551 raft_consensus.cc:2243] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.608546 8551 raft_consensus.cc:2272] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.608685 8545 tablet_bootstrap.cc:492] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.608758 8545 ts_tablet_manager.cc:1403] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:50.608819 8544 tablet_bootstrap.cc:492] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.608892 8544 ts_tablet_manager.cc:1403] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:50.609056 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]). Expected version 1 got 0
I20251028 02:59:50.609249 8545 raft_consensus.cc:359] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.609369 8545 raft_consensus.cc:385] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.609397 8545 raft_consensus.cc:740] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.609380 8544 raft_consensus.cc:359] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.609483 8544 raft_consensus.cc:385] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.609524 8544 raft_consensus.cc:740] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.609508 8545 consensus_queue.cc:260] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.609640 8544 consensus_queue.cc:260] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.609789 8545 ts_tablet_manager.cc:1434] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.609930 8544 ts_tablet_manager.cc:1434] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.609969 8545 tablet_bootstrap.cc:492] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.610097 8544 tablet_bootstrap.cc:492] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.610996 8545 tablet_bootstrap.cc:654] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.611317 8544 tablet_bootstrap.cc:654] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.611497 8549 raft_consensus.cc:493] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.611640 8549 raft_consensus.cc:515] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.612519 8546 tablet_bootstrap.cc:492] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.612618 8546 ts_tablet_manager.cc:1403] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:50.612946 8549 leader_election.cc:290] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:50.613103 8559 hybrid_clock.cc:648] HybridClock initialized: now 1761620390612585 us; error 576 us; skew 500 ppm
I20251028 02:59:50.613137 8546 raft_consensus.cc:359] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.613250 8546 raft_consensus.cc:385] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.613283 8546 raft_consensus.cc:740] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.613408 8546 consensus_queue.cc:260] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.613709 8546 ts_tablet_manager.cc:1434] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.613912 8546 tablet_bootstrap.cc:492] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.614902 8556 raft_consensus.cc:493] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.615005 8546 tablet_bootstrap.cc:654] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.615067 8556 raft_consensus.cc:515] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.615773 8559 webserver.cc:492] Webserver started at http://127.7.154.189:36469/ using document root <none> and password file <none>
I20251028 02:59:50.615923 8551 ts_tablet_manager.cc:1916] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.616328 8556 leader_election.cc:290] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.616520 8559 fs_manager.cc:362] Metadata directory not provided
I20251028 02:59:50.616593 8559 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 02:59:50.617954 8546 tablet_bootstrap.cc:492] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.618045 8546 ts_tablet_manager.cc:1403] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251028 02:59:50.618108 8551 ts_tablet_manager.cc:1929] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.618170 8551 log.cc:1199] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a3d108be7a734497a9619cd7c097b2e5
I20251028 02:59:50.618369 8551 ts_tablet_manager.cc:1950] T a3d108be7a734497a9619cd7c097b2e5 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.618618 8546 raft_consensus.cc:359] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.618721 8546 raft_consensus.cc:385] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.618762 8546 raft_consensus.cc:740] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.618901 8546 consensus_queue.cc:260] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.619175 8546 ts_tablet_manager.cc:1434] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.619175 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet f74e7c0e7d634679bb70cf3515aafe22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.619415 8546 tablet_bootstrap.cc:492] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.620452 8546 tablet_bootstrap.cc:654] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.621331 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a3d108be7a734497a9619cd7c097b2e5 (table default.Table_ddb39e36d68643c2909826b661af4128 [id=8f89769a43034f7d9f783602532e7fef]) successfully deleted
I20251028 02:59:50.621913 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]). Expected version 1 got 0
I20251028 02:59:50.622629 8550 tablet_replica.cc:333] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.623190 8550 raft_consensus.cc:2243] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.623347 8550 raft_consensus.cc:2272] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.623953 8550 ts_tablet_manager.cc:1916] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.624325 8559 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20251028 02:59:50.625141 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet bd3398b5e77d494bb821a2daf73fe593 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.625312 8552 tablet_replica.cc:333] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.625437 8552 raft_consensus.cc:2243] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.625550 8552 raft_consensus.cc:2272] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.626031 8552 ts_tablet_manager.cc:1916] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.626276 8550 ts_tablet_manager.cc:1929] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.626359 8550 log.cc:1199] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/f74e7c0e7d634679bb70cf3515aafe22
I20251028 02:59:50.626590 8550 ts_tablet_manager.cc:1950] T f74e7c0e7d634679bb70cf3515aafe22 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.627256 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet f74e7c0e7d634679bb70cf3515aafe22 (table default.Table_13e18d2c103343648ddade08c263f084 [id=93f5552c487f4ad49d4a1cdf54ea08e7]) successfully deleted
I20251028 02:59:50.627983 8544 tablet_bootstrap.cc:492] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.628082 8544 ts_tablet_manager.cc:1403] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20251028 02:59:50.628579 8544 raft_consensus.cc:359] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.628707 8544 raft_consensus.cc:385] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.628763 8544 raft_consensus.cc:740] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.628921 8544 consensus_queue.cc:260] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.629199 8544 ts_tablet_manager.cc:1434] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.629278 8552 ts_tablet_manager.cc:1929] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.629251 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d09229ca82eb49f2b4214b238c3ec097" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:50.629339 8552 log.cc:1199] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/bd3398b5e77d494bb821a2daf73fe593
I20251028 02:59:50.629396 8544 tablet_bootstrap.cc:492] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.629536 8552 ts_tablet_manager.cc:1950] T bd3398b5e77d494bb821a2daf73fe593 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.630211 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet bd3398b5e77d494bb821a2daf73fe593 (table default.Table_1f66803097e64e178e8ffeacb35d4faa [id=0baa5ac1f0d241d29fdb43f98433b9b7]) successfully deleted
I20251028 02:59:50.630448 8544 tablet_bootstrap.cc:654] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:50.630630 8354 leader_election.cc:343] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:50.631661 8628 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:50.633225 8559 fs_manager.cc:730] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20251028 02:59:50.633347 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2386ea3c08fe48c381c7fc2321ecbc86" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:50.633553 8421 raft_consensus.cc:2468] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:50.633433 8559 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data,/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
uuid: "aac264d6d2274d3283bd3251ab77e583"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
I20251028 02:59:50.634001 8559 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 02:59:50.634109 8220 leader_election.cc:304] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:50.634518 8556 raft_consensus.cc:2804] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.634531 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d09229ca82eb49f2b4214b238c3ec097" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:50.634615 8556 raft_consensus.cc:493] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.634660 8556 raft_consensus.cc:3060] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:50.635030 8356 leader_election.cc:343] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:50.635131 8356 leader_election.cc:304] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:50.635398 8616 raft_consensus.cc:2749] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:50.635783 8556 raft_consensus.cc:515] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.636157 8556 leader_election.cc:290] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.636627 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2386ea3c08fe48c381c7fc2321ecbc86" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:50.636744 8421 raft_consensus.cc:3060] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.636823 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2386ea3c08fe48c381c7fc2321ecbc86" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:50.636864 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2386ea3c08fe48c381c7fc2321ecbc86" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:50.637022 8148 raft_consensus.cc:2468] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:50.637866 8421 raft_consensus.cc:2468] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:50.638264 8220 leader_election.cc:304] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:50.638334 8544 tablet_bootstrap.cc:492] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.638419 8544 ts_tablet_manager.cc:1403] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20251028 02:59:50.638514 8546 tablet_bootstrap.cc:492] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.638559 8556 raft_consensus.cc:2804] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.638602 8546 ts_tablet_manager.cc:1403] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.001s
I20251028 02:59:50.638639 8556 raft_consensus.cc:697] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:50.638875 8556 consensus_queue.cc:237] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.638964 8544 raft_consensus.cc:359] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.639074 8544 raft_consensus.cc:385] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.639055 8546 raft_consensus.cc:359] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.639115 8544 raft_consensus.cc:740] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.639132 8546 raft_consensus.cc:385] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.639158 8546 raft_consensus.cc:740] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.639222 8544 consensus_queue.cc:260] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.639261 8546 consensus_queue.cc:260] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.639663 8546 ts_tablet_manager.cc:1434] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.639870 8546 tablet_bootstrap.cc:492] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.641057 8546 tablet_bootstrap.cc:654] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.642414 8545 tablet_bootstrap.cc:492] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.642516 8545 ts_tablet_manager.cc:1403] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.002s
I20251028 02:59:50.642614 7966 catalog_manager.cc:5649] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:50.643033 8545 raft_consensus.cc:359] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.643126 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 2386ea3c08fe48c381c7fc2321ecbc86 (table default.Table_5036a52b716149b2a336ce9c3a019ed1 [id=9d19f4b99b02446c8bbc1263a9b70c4e]). Expected version 2 got 0
I20251028 02:59:50.643146 8545 raft_consensus.cc:385] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.643187 8545 raft_consensus.cc:740] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.643347 8545 consensus_queue.cc:260] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.643630 8545 ts_tablet_manager.cc:1434] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.646571 8544 ts_tablet_manager.cc:1434] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.008s user 0.000s sys 0.001s
I20251028 02:59:50.646803 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 5444c480b4c84863bfb85f6367227005 (table default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]). Expected version 1 got 0
W20251028 02:59:50.647138 7960 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet 5444c480b4c84863bfb85f6367227005 (table default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 5444c480b4c84863bfb85f6367227005 consensus state has no leader
I20251028 02:59:50.648099 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bf33aa6089da40008abddb5fe2796251"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.648517 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf33aa6089da40008abddb5fe2796251 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.649255 8544 tablet_bootstrap.cc:492] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.650250 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 5444c480b4c84863bfb85f6367227005 (table default.Table_2204fdfdab02441eb98f08410e42c935 [id=eb9bbde4c5c54a928e854dab13c17ee5]). Expected version 1 got 0
I20251028 02:59:50.650947 8544 tablet_bootstrap.cc:654] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.652170 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ff1533cf65f64af7919ffc1aeadd1cac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.652632 8544 tablet_bootstrap.cc:492] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
W20251028 02:59:50.652652 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff1533cf65f64af7919ffc1aeadd1cac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.652717 8544 ts_tablet_manager.cc:1403] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251028 02:59:50.653141 8544 raft_consensus.cc:359] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.653235 8544 raft_consensus.cc:385] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.653261 8544 raft_consensus.cc:740] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.653357 8544 consensus_queue.cc:260] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.653615 8544 ts_tablet_manager.cc:1434] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.653785 8544 tablet_bootstrap.cc:492] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.654678 8544 tablet_bootstrap.cc:654] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.655649 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_db55a7fbf01e47f4b7f596581c362990"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.656016 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db55a7fbf01e47f4b7f596581c362990 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.656041 8546 tablet_bootstrap.cc:492] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.656143 8546 ts_tablet_manager.cc:1403] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 02:59:50.656658 8546 raft_consensus.cc:359] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.656764 8546 raft_consensus.cc:385] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.656806 8546 raft_consensus.cc:740] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.656934 8546 consensus_queue.cc:260] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.657244 8546 ts_tablet_manager.cc:1434] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.658039 8562 raft_consensus.cc:493] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.658123 8546 tablet_bootstrap.cc:492] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.658136 8562 raft_consensus.cc:515] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.659149 8546 tablet_bootstrap.cc:654] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.659442 8562 leader_election.cc:290] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.660281 8544 tablet_bootstrap.cc:492] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.660362 8544 ts_tablet_manager.cc:1403] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20251028 02:59:50.660810 8544 raft_consensus.cc:359] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.660913 8544 raft_consensus.cc:385] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.660953 8544 raft_consensus.cc:740] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.661099 8544 consensus_queue.cc:260] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.661373 8544 ts_tablet_manager.cc:1434] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:50.661559 8544 tablet_bootstrap.cc:492] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.661906 8546 tablet_bootstrap.cc:492] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.661986 8546 ts_tablet_manager.cc:1403] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251028 02:59:50.662541 8544 tablet_bootstrap.cc:654] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.662478 8546 raft_consensus.cc:359] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.662586 8546 raft_consensus.cc:385] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.662624 8546 raft_consensus.cc:740] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.662765 8546 consensus_queue.cc:260] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.663067 8546 ts_tablet_manager.cc:1434] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.663691 8546 tablet_bootstrap.cc:492] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.664685 8546 tablet_bootstrap.cc:654] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.670156 8631 raft_consensus.cc:493] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.670276 8631 raft_consensus.cc:515] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.670449 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85a0ee11b35545cca8db8754c0b6b7d7" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:50.670583 8285 raft_consensus.cc:2468] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:50.663245 8616 raft_consensus.cc:493] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.670773 8616 raft_consensus.cc:515] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.670965 8083 leader_election.cc:304] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.671202 8616 leader_election.cc:290] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:50.671241 8562 raft_consensus.cc:2804] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.671334 8562 raft_consensus.cc:493] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.671388 8562 raft_consensus.cc:3060] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.671671 8631 leader_election.cc:290] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.672164 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85a0ee11b35545cca8db8754c0b6b7d7" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:50.672274 8285 raft_consensus.cc:2468] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:50.672474 8544 tablet_bootstrap.cc:492] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.672580 8544 ts_tablet_manager.cc:1403] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:50.672600 8356 leader_election.cc:304] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:50.672592 8562 raft_consensus.cc:515] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.672768 8616 raft_consensus.cc:2804] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.672833 8616 raft_consensus.cc:493] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.672873 8616 raft_consensus.cc:3060] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.672950 8562 leader_election.cc:290] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.673084 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85a0ee11b35545cca8db8754c0b6b7d7" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:50.673151 8544 raft_consensus.cc:359] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.673233 8148 raft_consensus.cc:2393] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7e4cf694f71438298dd32a346abcf75 in current term 1: Already voted for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in this term.
I20251028 02:59:50.673321 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d09229ca82eb49f2b4214b238c3ec097" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:50.673403 8147 raft_consensus.cc:2468] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:50.673847 8218 leader_election.cc:304] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.673869 8616 raft_consensus.cc:515] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.673245 8544 raft_consensus.cc:385] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.673970 8544 raft_consensus.cc:740] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.674037 8631 raft_consensus.cc:2804] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.674093 8631 raft_consensus.cc:493] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.674131 8631 raft_consensus.cc:3060] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.674088 8544 consensus_queue.cc:260] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.674218 8616 leader_election.cc:290] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:50.674402 8544 ts_tablet_manager.cc:1434] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:50.675029 8265 tablet_service.cc:1505] Processing CreateTablet for tablet da5a607a1e7f4ee9b7468b7a12b5a484 (DEFAULT_TABLE table=default.Table_bf33aa6089da40008abddb5fe2796251 [id=dfeb1ac7ead5405c811d04273658497b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.675271 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da5a607a1e7f4ee9b7468b7a12b5a484. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.675526 8631 raft_consensus.cc:515] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.675894 8631 leader_election.cc:290] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.676646 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ef5a7a00ec9e47349f4b15f619f609f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.676733 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4beb9b222fd14525832fb1ff5029e701"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.677076 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4beb9b222fd14525832fb1ff5029e701 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.677309 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c022e2c53bc7477f9709e148a559ef5b" } new_table_name: "default.Table_a7f1f8afc727489faff5840c8881f773" modify_external_catalogs: true
I20251028 02:59:50.677791 8398 tablet_service.cc:1505] Processing CreateTablet for tablet da5a607a1e7f4ee9b7468b7a12b5a484 (DEFAULT_TABLE table=default.Table_bf33aa6089da40008abddb5fe2796251 [id=dfeb1ac7ead5405c811d04273658497b]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:50.677076 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef5a7a00ec9e47349f4b15f619f609f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.678036 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da5a607a1e7f4ee9b7468b7a12b5a484. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.678037 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_dff2d248a846455eb7bd3f3648f7ceee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.678337 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dff2d248a846455eb7bd3f3648f7ceee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.678891 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ca53be73e3384c6ebb65c41f31a6a4b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.679116 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca53be73e3384c6ebb65c41f31a6a4b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.679884 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d09229ca82eb49f2b4214b238c3ec097" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:50.679960 8545 tablet_bootstrap.cc:492] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.680001 8420 raft_consensus.cc:3060] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.680908 8545 tablet_bootstrap.cc:654] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.680951 8420 raft_consensus.cc:2468] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:50.681527 8220 leader_election.cc:304] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:50.682693 8127 tablet_service.cc:1505] Processing CreateTablet for tablet da5a607a1e7f4ee9b7468b7a12b5a484 (DEFAULT_TABLE table=default.Table_bf33aa6089da40008abddb5fe2796251 [id=dfeb1ac7ead5405c811d04273658497b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.682950 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da5a607a1e7f4ee9b7468b7a12b5a484. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.683929 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85a0ee11b35545cca8db8754c0b6b7d7" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:50.684057 8285 raft_consensus.cc:3060] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.684811 8544 tablet_bootstrap.cc:492] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.684857 8616 raft_consensus.cc:493] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.684952 8616 raft_consensus.cc:515] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.685052 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85a0ee11b35545cca8db8754c0b6b7d7" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:50.685103 8285 raft_consensus.cc:2468] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:50.685372 8616 leader_election.cc:290] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:50.685544 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d09229ca82eb49f2b4214b238c3ec097" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:50.685676 8421 raft_consensus.cc:2376] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1. Re-sending same reply.
I20251028 02:59:50.685869 8631 raft_consensus.cc:2804] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.685918 8544 tablet_bootstrap.cc:654] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.685911 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "caff8b9ed8a54ccbbcdfac2d62bc2a71" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:50.685953 8631 raft_consensus.cc:697] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:50.686014 8284 raft_consensus.cc:2468] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:50.686082 8631 consensus_queue.cc:237] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.686391 8356 leader_election.cc:304] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
W20251028 02:59:50.686635 7954 catalog_manager.cc:5101] could not convert tablet's 2386ea3c08fe48c381c7fc2321ecbc86 schema from PB: OK
I20251028 02:59:50.687593 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d09229ca82eb49f2b4214b238c3ec097" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:50.687714 8147 raft_consensus.cc:3060] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.687800 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85a0ee11b35545cca8db8754c0b6b7d7" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:50.687929 8148 raft_consensus.cc:2393] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d7e4cf694f71438298dd32a346abcf75 in current term 1: Already voted for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in this term.
I20251028 02:59:50.688169 8356 leader_election.cc:304] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:50.688324 8549 raft_consensus.cc:2804] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.688540 8549 raft_consensus.cc:697] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:50.688712 8147 raft_consensus.cc:2468] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:50.688922 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "caff8b9ed8a54ccbbcdfac2d62bc2a71" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:50.689030 8146 raft_consensus.cc:2468] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:50.689741 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2204fdfdab02441eb98f08410e42c935" } new_table_name: "default.Table_c1124b12dc1640bcbb89fa3d6c624c75" modify_external_catalogs: true
I20251028 02:59:50.689862 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_49d0ca6e0aa54b309562cc456bcf1be5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.690248 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49d0ca6e0aa54b309562cc456bcf1be5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.692164 8546 tablet_bootstrap.cc:492] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.692253 8546 ts_tablet_manager.cc:1403] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.001s
I20251028 02:59:50.692728 8546 raft_consensus.cc:359] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.692842 8546 raft_consensus.cc:385] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.692873 8546 raft_consensus.cc:740] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.689770 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b4b24abe267c451f95fc63bf0234d06c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.692996 8546 consensus_queue.cc:260] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:50.693274 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b4b24abe267c451f95fc63bf0234d06c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.693342 8546 ts_tablet_manager.cc:1434] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.693535 8546 tablet_bootstrap.cc:492] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.694540 8546 tablet_bootstrap.cc:654] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.688841 8549 consensus_queue.cc:237] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.697670 8544 tablet_bootstrap.cc:492] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.697768 8544 ts_tablet_manager.cc:1403] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20251028 02:59:50.698292 8544 raft_consensus.cc:359] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.698413 8544 raft_consensus.cc:385] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.698442 8544 raft_consensus.cc:740] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.698554 8544 consensus_queue.cc:260] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.698812 8544 ts_tablet_manager.cc:1434] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.699213 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d656e1f10f1e4dba925e8578af15b760" } modify_external_catalogs: true
I20251028 02:59:50.699342 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d656e1f10f1e4dba925e8578af15b760" } modify_external_catalogs: true
I20251028 02:59:50.699506 8562 raft_consensus.cc:493] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.699570 8562 raft_consensus.cc:515] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.699780 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_dbd38ab02bf5402aadd9c1bc0dfb7ba2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.699867 8562 leader_election.cc:290] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:50.700150 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dbd38ab02bf5402aadd9c1bc0dfb7ba2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.700268 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "206b71b0f10c42fd81f291a27fffc8a2" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:50.700394 8284 raft_consensus.cc:2468] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:50.700675 8083 leader_election.cc:304] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.700821 8562 raft_consensus.cc:2804] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.700862 8562 raft_consensus.cc:493] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.700888 8562 raft_consensus.cc:3060] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.701521 8419 raft_consensus.cc:1275] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:50.701861 8562 raft_consensus.cc:515] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.702164 8562 leader_election.cc:290] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.702188 8631 consensus_queue.cc:1048] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:50.700723 8559 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 02:59:50.702595 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "206b71b0f10c42fd81f291a27fffc8a2" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:50.702702 8284 raft_consensus.cc:3060] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.692718 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85a0ee11b35545cca8db8754c0b6b7d7" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:50.703186 8559 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251028 02:59:50.703406 8559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 02:59:50.703517 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "206b71b0f10c42fd81f291a27fffc8a2" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:50.703622 8420 raft_consensus.cc:2468] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:50.703850 8284 raft_consensus.cc:2468] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:50.703855 8083 leader_election.cc:304] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:50.704030 8562 raft_consensus.cc:2749] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20251028 02:59:50.704342 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "206b71b0f10c42fd81f291a27fffc8a2" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:50.704438 8419 raft_consensus.cc:3060] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.705960 8419 raft_consensus.cc:2468] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:50.706552 8083 leader_election.cc:304] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.706748 8562 raft_consensus.cc:2804] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.706957 8562 raft_consensus.cc:697] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:50.707222 8562 consensus_queue.cc:237] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.708316 8546 tablet_bootstrap.cc:492] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.708415 8546 ts_tablet_manager.cc:1403] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.001s
I20251028 02:59:50.708899 8546 raft_consensus.cc:359] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.709004 8546 raft_consensus.cc:385] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.709051 8546 raft_consensus.cc:740] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.709048 8562 raft_consensus.cc:493] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.709115 8562 raft_consensus.cc:515] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.709169 8546 consensus_queue.cc:260] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.709425 8562 leader_election.cc:290] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.709460 8546 ts_tablet_manager.cc:1434] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251028 02:59:50.709650 8546 tablet_bootstrap.cc:492] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.710680 8546 tablet_bootstrap.cc:654] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.711787 8546 tablet_bootstrap.cc:492] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.711882 8546 ts_tablet_manager.cc:1403] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20251028 02:59:50.712330 8127 tablet_service.cc:1505] Processing CreateTablet for tablet ae2927be5f3a440286ee0fe5ce818018 (DEFAULT_TABLE table=default.Table_ff1533cf65f64af7919ffc1aeadd1cac [id=f490a46696124e7ea69612e94d7321d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.712352 8546 raft_consensus.cc:359] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.712451 8546 raft_consensus.cc:385] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.712481 8546 raft_consensus.cc:740] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.712481 8146 raft_consensus.cc:3060] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.712599 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae2927be5f3a440286ee0fe5ce818018. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.712589 8546 consensus_queue.cc:260] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.712908 8546 ts_tablet_manager.cc:1434] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251028 02:59:50.713104 8546 tablet_bootstrap.cc:492] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.713577 8146 raft_consensus.cc:1275] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:50.714471 8546 tablet_bootstrap.cc:654] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.714819 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 0876f8b4ee634d9d92efe9613b3495be (DEFAULT_TABLE table=default.Table_db55a7fbf01e47f4b7f596581c362990 [id=7a2a6ab3c9744942a9413c6cb057f527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.715063 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0876f8b4ee634d9d92efe9613b3495be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.715251 8631 consensus_queue.cc:1048] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:50.717084 8544 tablet_bootstrap.cc:492] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.717671 8559 rpc_server.cc:307] RPC server started. Bound to: 127.7.154.189:45473
I20251028 02:59:50.718421 8544 tablet_bootstrap.cc:654] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.718797 8559 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/info.pb
I20251028 02:59:50.719982 7786 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu as pid 8559
I20251028 02:59:50.722193 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29396aa23c734dc78983623aca442ba6" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:50.722345 8284 raft_consensus.cc:2468] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:50.722697 8083 leader_election.cc:304] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.722896 8562 raft_consensus.cc:2804] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.722955 8562 raft_consensus.cc:493] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.722992 8562 raft_consensus.cc:3060] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.723412 8549 raft_consensus.cc:493] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.723517 8549 raft_consensus.cc:515] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.723930 8549 leader_election.cc:290] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:50.724026 8562 raft_consensus.cc:515] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.724349 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0a3d0d8a0e5e4e34b97230c97b5f27b5" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:50.724502 8147 raft_consensus.cc:2468] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:50.724546 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 0876f8b4ee634d9d92efe9613b3495be (DEFAULT_TABLE table=default.Table_db55a7fbf01e47f4b7f596581c362990 [id=7a2a6ab3c9744942a9413c6cb057f527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.724362 8562 leader_election.cc:290] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.724774 8265 tablet_service.cc:1505] Processing CreateTablet for tablet ae2927be5f3a440286ee0fe5ce818018 (DEFAULT_TABLE table=default.Table_ff1533cf65f64af7919ffc1aeadd1cac [id=f490a46696124e7ea69612e94d7321d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.724787 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29396aa23c734dc78983623aca442ba6" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:50.724937 8284 raft_consensus.cc:3060] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.725076 8354 leader_election.cc:304] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.725206 8398 tablet_service.cc:1505] Processing CreateTablet for tablet ae2927be5f3a440286ee0fe5ce818018 (DEFAULT_TABLE table=default.Table_ff1533cf65f64af7919ffc1aeadd1cac [id=f490a46696124e7ea69612e94d7321d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.725291 8544 tablet_bootstrap.cc:492] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.725351 8549 raft_consensus.cc:2804] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.725378 8544 ts_tablet_manager.cc:1403] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20251028 02:59:50.725404 8549 raft_consensus.cc:493] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.725445 8549 raft_consensus.cc:3060] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.725462 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae2927be5f3a440286ee0fe5ce818018. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.725920 8284 raft_consensus.cc:2468] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:50.725899 8544 raft_consensus.cc:359] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.725994 8544 raft_consensus.cc:385] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.726024 8544 raft_consensus.cc:740] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.724831 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0876f8b4ee634d9d92efe9613b3495be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.726161 8544 consensus_queue.cc:260] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.726383 8549 raft_consensus.cc:515] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.726480 8544 ts_tablet_manager.cc:1434] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.726655 8544 tablet_bootstrap.cc:492] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.726764 8549 leader_election.cc:290] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:50.727237 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0a3d0d8a0e5e4e34b97230c97b5f27b5" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:50.727380 8147 raft_consensus.cc:3060] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.727507 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 0876f8b4ee634d9d92efe9613b3495be (DEFAULT_TABLE table=default.Table_db55a7fbf01e47f4b7f596581c362990 [id=7a2a6ab3c9744942a9413c6cb057f527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.727734 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0876f8b4ee634d9d92efe9613b3495be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.728291 8147 raft_consensus.cc:2468] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:50.728559 8083 leader_election.cc:304] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.728703 8562 raft_consensus.cc:2804] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.728713 8354 leader_election.cc:304] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.728758 8562 raft_consensus.cc:697] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:50.728889 8549 raft_consensus.cc:2804] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.728894 8562 consensus_queue.cc:237] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.728992 8544 tablet_bootstrap.cc:654] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.729238 8549 raft_consensus.cc:697] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:50.729413 8549 consensus_queue.cc:237] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.729823 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29396aa23c734dc78983623aca442ba6" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:50.729856 8706 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.154.189:45473 every 8 connection(s)
I20251028 02:59:50.729928 8418 raft_consensus.cc:3060] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.730136 8631 raft_consensus.cc:493] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.730221 8631 raft_consensus.cc:515] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.730556 8631 leader_election.cc:290] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.730829 8418 raft_consensus.cc:2468] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:50.731057 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3df041f16a0448839351bbc768633b69" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:50.731161 8417 raft_consensus.cc:2468] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:50.731413 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0a3d0d8a0e5e4e34b97230c97b5f27b5" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:50.731550 8284 raft_consensus.cc:2468] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:50.731424 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae2927be5f3a440286ee0fe5ce818018. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.731817 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3df041f16a0448839351bbc768633b69" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:50.731937 8147 raft_consensus.cc:2468] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:50.732546 8546 tablet_bootstrap.cc:492] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.732630 8546 ts_tablet_manager.cc:1403] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.001s
I20251028 02:59:50.733103 8546 raft_consensus.cc:359] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.733191 8546 raft_consensus.cc:385] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.733217 8546 raft_consensus.cc:740] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.733321 8546 consensus_queue.cc:260] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.733582 8546 ts_tablet_manager.cc:1434] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.733758 8546 tablet_bootstrap.cc:492] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.733793 8707 sys_catalog.cc:263] Verifying existing consensus state
I20251028 02:59:50.731431 8220 leader_election.cc:304] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:50.734403 8553 raft_consensus.cc:2804] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.734464 8553 raft_consensus.cc:493] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.734515 8553 raft_consensus.cc:3060] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.734803 8546 tablet_bootstrap.cc:654] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.735478 8553 raft_consensus.cc:515] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.735823 8553 leader_election.cc:290] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.736225 8707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Bootstrap starting.
I20251028 02:59:50.736387 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3df041f16a0448839351bbc768633b69" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:50.736510 8417 raft_consensus.cc:3060] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.737545 8417 raft_consensus.cc:2468] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:50.737839 8220 leader_election.cc:304] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:50.738117 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_db55a7fbf01e47f4b7f596581c362990" } new_table_name: "default.Table_b026ff28518f49aea62b20dd70917a7a" modify_external_catalogs: true
I20251028 02:59:50.739027 8553 raft_consensus.cc:2804] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.739112 8553 raft_consensus.cc:697] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:50.739244 8553 consensus_queue.cc:237] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.695106 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85a0ee11b35545cca8db8754c0b6b7d7" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:50.695895 8545 tablet_bootstrap.cc:492] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.740922 8545 ts_tablet_manager.cc:1403] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.061s user 0.001s sys 0.000s
I20251028 02:59:50.741441 8545 raft_consensus.cc:359] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.741540 8545 raft_consensus.cc:385] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.741569 8545 raft_consensus.cc:740] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.741683 8545 consensus_queue.cc:260] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:50.741756 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5444c480b4c84863bfb85f6367227005 consensus state has no leader
I20251028 02:59:50.741958 8545 ts_tablet_manager.cc:1434] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.742008 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3396d9a29e3148beaa9e1fadc46527d0" } new_table_name: "default.Table_1cdf4b1794504307906ac65e49eb2c71" modify_external_catalogs: true
I20251028 02:59:50.742113 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ff1533cf65f64af7919ffc1aeadd1cac" } modify_external_catalogs: true
I20251028 02:59:50.742157 8545 tablet_bootstrap.cc:492] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.742195 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ff1533cf65f64af7919ffc1aeadd1cac" } modify_external_catalogs: true
I20251028 02:59:50.743175 8545 tablet_bootstrap.cc:654] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.744184 8707 log.cc:826] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Log is configured to *not* fsync() on all Append() calls
I20251028 02:59:50.744944 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0a3d0d8a0e5e4e34b97230c97b5f27b5" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:50.745079 8285 raft_consensus.cc:3060] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.746083 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5036a52b716149b2a336ce9c3a019ed1" } modify_external_catalogs: true
I20251028 02:59:50.746120 8285 raft_consensus.cc:2468] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:50.746191 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5036a52b716149b2a336ce9c3a019ed1" } modify_external_catalogs: true
I20251028 02:59:50.747493 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29396aa23c734dc78983623aca442ba6" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:50.747615 8419 raft_consensus.cc:2376] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1. Re-sending same reply.
I20251028 02:59:50.749794 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3df041f16a0448839351bbc768633b69" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:50.749933 8146 raft_consensus.cc:3060] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.751048 8146 raft_consensus.cc:2468] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:50.751510 8649 mvcc.cc:204] Tried to move back new op lower bound from 7215597120291786752 to 7215597120060014592. Current Snapshot: MvccSnapshot[applied={T|T < 7215597120291786752}]
I20251028 02:59:50.752004 8707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 02:59:50.752418 8707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Bootstrap complete.
I20251028 02:59:50.753333 8546 tablet_bootstrap.cc:492] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.753417 8546 ts_tablet_manager.cc:1403] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.001s
I20251028 02:59:50.753885 8546 raft_consensus.cc:359] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.753993 8546 raft_consensus.cc:385] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.754022 8546 raft_consensus.cc:740] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.754141 8546 consensus_queue.cc:260] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.754423 8546 ts_tablet_manager.cc:1434] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.754601 8546 tablet_bootstrap.cc:492] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.755362 8710 mvcc.cc:204] Tried to move back new op lower bound from 7215597120291786752 to 7215597120060014592. Current Snapshot: MvccSnapshot[applied={T|T < 7215597120291786752}]
I20251028 02:59:50.756125 8707 raft_consensus.cc:359] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:50.756541 8707 raft_consensus.cc:740] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aac264d6d2274d3283bd3251ab77e583, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.757072 8707 consensus_queue.cc:260] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:50.757756 8546 tablet_bootstrap.cc:654] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.758499 8545 tablet_bootstrap.cc:492] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.758605 8545 ts_tablet_manager.cc:1403] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 02:59:50.759092 8545 raft_consensus.cc:359] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.759207 8545 raft_consensus.cc:385] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.759241 8545 raft_consensus.cc:740] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.759383 8545 consensus_queue.cc:260] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.759662 8545 ts_tablet_manager.cc:1434] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.759742 8707 sys_catalog.cc:565] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 02:59:50.759888 8545 tablet_bootstrap.cc:492] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.760964 8545 tablet_bootstrap.cc:654] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.763978 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_79fe71e0a8b243c786b93a2345f7d1b8" } new_table_name: "default.Table_43b9d6e2a6af4b25ae6e80ecccbe4392" modify_external_catalogs: true
I20251028 02:59:50.766840 8727 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:50.768258 8727 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:50.770854 8739 catalog_manager.cc:1269] Loaded cluster ID: a891460fa9a9449bb0459a49a032f9be
I20251028 02:59:50.771020 8739 catalog_manager.cc:1562] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: loading cluster ID for follower catalog manager: success
I20251028 02:59:50.773090 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 36d7cf962de845cfaca691be18a1bd63 (DEFAULT_TABLE table=default.Table_4beb9b222fd14525832fb1ff5029e701 [id=4a127e52978a4c8ea8a947478ce5996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.773417 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d7cf962de845cfaca691be18a1bd63. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:50.774084 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0876f8b4ee634d9d92efe9613b3495be consensus state has no leader
I20251028 02:59:50.774258 8739 catalog_manager.cc:1584] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: acquiring CA information for follower catalog manager: success
I20251028 02:59:50.774269 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet ae2927be5f3a440286ee0fe5ce818018 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.775553 8126 tablet_service.cc:1505] Processing CreateTablet for tablet de87fed15e1f4e41b0d962aa858e3128 (DEFAULT_TABLE table=default.Table_ca53be73e3384c6ebb65c41f31a6a4b1 [id=b25233dc76034f46a52c13bb1010e305]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.775802 8739 catalog_manager.cc:1612] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251028 02:59:50.775821 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de87fed15e1f4e41b0d962aa858e3128. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:50.778524 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d09229ca82eb49f2b4214b238c3ec097 consensus state has no leader
I20251028 02:59:50.779168 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 5444c480b4c84863bfb85f6367227005 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.780066 7964 catalog_manager.cc:5649] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:50.780277 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 85a0ee11b35545cca8db8754c0b6b7d7 (table default.Table_3396d9a29e3148beaa9e1fadc46527d0 [id=b34ab58e43e943909c9bdd79ad69e639]). Expected version 2 got 0
W20251028 02:59:50.780915 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85a0ee11b35545cca8db8754c0b6b7d7 consensus state has no leader
I20251028 02:59:50.781423 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bf33aa6089da40008abddb5fe2796251" } modify_external_catalogs: true
I20251028 02:59:50.781526 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bf33aa6089da40008abddb5fe2796251" } modify_external_catalogs: true
I20251028 02:59:50.783082 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 70e625899ec546f7add9948e59d289af (DEFAULT_TABLE table=default.Table_dff2d248a846455eb7bd3f3648f7ceee [id=77dd1eb6df0747bcab57b157ab8a850e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.783387 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e625899ec546f7add9948e59d289af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.788111 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 9b0abfe5364749bd86b9916480828ffd (DEFAULT_TABLE table=default.Table_ef5a7a00ec9e47349f4b15f619f609f7 [id=b73cb9b72fab4a68946bb2ce1ddcf026]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.788383 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b0abfe5364749bd86b9916480828ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.791530 8726 tablet.cc:1722] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:50.791610 8265 tablet_service.cc:1505] Processing CreateTablet for tablet 70e625899ec546f7add9948e59d289af (DEFAULT_TABLE table=default.Table_dff2d248a846455eb7bd3f3648f7ceee [id=77dd1eb6df0747bcab57b157ab8a850e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.791896 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e625899ec546f7add9948e59d289af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.793198 8545 tablet_bootstrap.cc:492] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.793290 8545 ts_tablet_manager.cc:1403] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20251028 02:59:50.794047 8545 raft_consensus.cc:359] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.794157 8545 raft_consensus.cc:385] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.794199 8545 raft_consensus.cc:740] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.794341 8545 consensus_queue.cc:260] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.794538 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 70e625899ec546f7add9948e59d289af (DEFAULT_TABLE table=default.Table_dff2d248a846455eb7bd3f3648f7ceee [id=77dd1eb6df0747bcab57b157ab8a850e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.794622 8545 ts_tablet_manager.cc:1434] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:50.794807 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70e625899ec546f7add9948e59d289af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.797289 8545 tablet_bootstrap.cc:492] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.797477 8546 tablet_bootstrap.cc:492] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.797564 8546 ts_tablet_manager.cc:1403] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.001s
I20251028 02:59:50.798038 8546 raft_consensus.cc:359] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.798161 8546 raft_consensus.cc:385] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.798192 8546 raft_consensus.cc:740] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.798259 8545 tablet_bootstrap.cc:654] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.798311 8546 consensus_queue.cc:260] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.798588 8546 ts_tablet_manager.cc:1434] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251028 02:59:50.798759 8546 tablet_bootstrap.cc:492] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.800753 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 36d7cf962de845cfaca691be18a1bd63 (DEFAULT_TABLE table=default.Table_4beb9b222fd14525832fb1ff5029e701 [id=4a127e52978a4c8ea8a947478ce5996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.801039 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d7cf962de845cfaca691be18a1bd63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.801059 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 36d7cf962de845cfaca691be18a1bd63 (DEFAULT_TABLE table=default.Table_4beb9b222fd14525832fb1ff5029e701 [id=4a127e52978a4c8ea8a947478ce5996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.801084 8260 tablet_service.cc:1505] Processing CreateTablet for tablet de87fed15e1f4e41b0d962aa858e3128 (DEFAULT_TABLE table=default.Table_ca53be73e3384c6ebb65c41f31a6a4b1 [id=b25233dc76034f46a52c13bb1010e305]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.801301 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36d7cf962de845cfaca691be18a1bd63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.802263 8265 tablet_service.cc:1505] Processing CreateTablet for tablet 9b0abfe5364749bd86b9916480828ffd (DEFAULT_TABLE table=default.Table_ef5a7a00ec9e47349f4b15f619f609f7 [id=b73cb9b72fab4a68946bb2ce1ddcf026]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.802549 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 9b0abfe5364749bd86b9916480828ffd (DEFAULT_TABLE table=default.Table_ef5a7a00ec9e47349f4b15f619f609f7 [id=b73cb9b72fab4a68946bb2ce1ddcf026]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.802887 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b0abfe5364749bd86b9916480828ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.803395 8400 tablet_service.cc:1505] Processing CreateTablet for tablet de87fed15e1f4e41b0d962aa858e3128 (DEFAULT_TABLE table=default.Table_ca53be73e3384c6ebb65c41f31a6a4b1 [id=b25233dc76034f46a52c13bb1010e305]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.803588 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de87fed15e1f4e41b0d962aa858e3128. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.805100 8546 tablet_bootstrap.cc:654] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.688668 8648 raft_consensus.cc:2804] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.806278 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b0abfe5364749bd86b9916480828ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.806499 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de87fed15e1f4e41b0d962aa858e3128. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.806416 8648 raft_consensus.cc:493] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.806640 8648 raft_consensus.cc:3060] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.807650 8648 raft_consensus.cc:515] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.808818 8648 leader_election.cc:290] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:50.810158 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "caff8b9ed8a54ccbbcdfac2d62bc2a71" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:50.810292 8285 raft_consensus.cc:3060] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.810520 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ae2927be5f3a440286ee0fe5ce818018
I20251028 02:59:50.810967 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ef5a7a00ec9e47349f4b15f619f609f7" } modify_external_catalogs: true
I20251028 02:59:50.811094 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ef5a7a00ec9e47349f4b15f619f609f7" } modify_external_catalogs: true
I20251028 02:59:50.811192 8285 raft_consensus.cc:2468] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:50.812244 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 29396aa23c734dc78983623aca442ba6
I20251028 02:59:50.812561 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.813452 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.808375 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "caff8b9ed8a54ccbbcdfac2d62bc2a71" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:50.815042 8146 raft_consensus.cc:3060] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.816066 8146 raft_consensus.cc:2468] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:50.816524 8354 leader_election.cc:304] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.817183 8546 tablet_bootstrap.cc:492] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.817279 8546 ts_tablet_manager.cc:1403] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.002s
I20251028 02:59:50.817763 8546 raft_consensus.cc:359] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.817889 8546 raft_consensus.cc:385] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.817921 8546 raft_consensus.cc:740] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.818027 8546 consensus_queue.cc:260] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.818102 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5444c480b4c84863bfb85f6367227005
I20251028 02:59:50.818303 8546 ts_tablet_manager.cc:1434] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.818364 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 5444c480b4c84863bfb85f6367227005 deleted and replaced by tablet c5a698db1042424bad62bce674b2c44d
I20251028 02:59:50.818486 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet c4b5b1d7c738413f80b8aed09b597c60 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:50.820293 8562 raft_consensus.cc:493] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.820396 8562 raft_consensus.cc:515] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:50.820451 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 0876f8b4ee634d9d92efe9613b3495be (table default.Table_b026ff28518f49aea62b20dd70917a7a [id=7a2a6ab3c9744942a9413c6cb057f527]). Expected version 1 got 0
I20251028 02:59:50.820552 7965 catalog_manager.cc:5649] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:50.820725 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet d09229ca82eb49f2b4214b238c3ec097 (table default.Table_a7f1f8afc727489faff5840c8881f773 [id=7fe387bad5e740429e327d42f1efde22]). Expected version 1 got 0
I20251028 02:59:50.820772 8562 leader_election.cc:290] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:50.820874 7965 catalog_manager.cc:5649] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:50.821031 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]). Expected version 1 got 0
I20251028 02:59:50.821518 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4b5b1d7c738413f80b8aed09b597c60" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:50.821545 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4b5b1d7c738413f80b8aed09b597c60" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:50.821640 8285 raft_consensus.cc:2468] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:50.821677 8419 raft_consensus.cc:2468] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:50.821918 8083 leader_election.cc:304] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.822088 8562 raft_consensus.cc:2804] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.822145 8562 raft_consensus.cc:493] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.822186 8562 raft_consensus.cc:3060] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.822289 8546 tablet_bootstrap.cc:492] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.823210 8562 raft_consensus.cc:515] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:50.823575 8546 tablet_bootstrap.cc:654] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.823581 8562 leader_election.cc:290] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:50.824288 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4b5b1d7c738413f80b8aed09b597c60" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:50.824429 8419 raft_consensus.cc:3060] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.825007 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 5444c480b4c84863bfb85f6367227005 with delete_type TABLET_DATA_DELETED (replaced by tablet c5a698db1042424bad62bce674b2c44d) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.825176 8550 tablet_replica.cc:333] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.825311 8550 raft_consensus.cc:2243] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.825424 8550 raft_consensus.cc:2272] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.825487 8419 raft_consensus.cc:2468] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:50.825891 8550 ts_tablet_manager.cc:1916] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.825968 8083 leader_election.cc:304] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.826195 8562 raft_consensus.cc:2804] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.826260 8562 raft_consensus.cc:697] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:50.826373 8562 consensus_queue.cc:237] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:50.828051 8550 ts_tablet_manager.cc:1929] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.828117 8550 log.cc:1199] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/5444c480b4c84863bfb85f6367227005
I20251028 02:59:50.828325 8550 ts_tablet_manager.cc:1950] T 5444c480b4c84863bfb85f6367227005 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.828825 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 29396aa23c734dc78983623aca442ba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.829012 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 5444c480b4c84863bfb85f6367227005 (table default.Table_c1124b12dc1640bcbb89fa3d6c624c75 [id=eb9bbde4c5c54a928e854dab13c17ee5]) successfully deleted
I20251028 02:59:50.829221 8550 tablet_replica.cc:333] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.829332 8550 raft_consensus.cc:2243] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:50.829448 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c4b5b1d7c738413f80b8aed09b597c60" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:50.829571 8550 pending_rounds.cc:70] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:50.829568 8285 raft_consensus.cc:3060] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.829663 8550 pending_rounds.cc:77] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597120427085824 op_type: NO_OP noop_request { }
I20251028 02:59:50.829735 8550 raft_consensus.cc:2889] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:50.829857 8550 raft_consensus.cc:2272] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.830096 8648 raft_consensus.cc:2804] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.830271 8648 raft_consensus.cc:697] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:50.830410 8648 consensus_queue.cc:237] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.830638 8285 raft_consensus.cc:2468] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:50.831905 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 29396aa23c734dc78983623aca442ba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.832142 8551 tablet_replica.cc:333] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.832165 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 5444c480b4c84863bfb85f6367227005 with delete_type TABLET_DATA_DELETED (replaced by tablet c5a698db1042424bad62bce674b2c44d) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.832284 8551 raft_consensus.cc:2243] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.832381 8551 raft_consensus.cc:2272] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.832962 8551 ts_tablet_manager.cc:1916] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.832998 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet ae2927be5f3a440286ee0fe5ce818018 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.833168 8552 tablet_replica.cc:333] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.833302 8552 raft_consensus.cc:2243] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.833413 8552 raft_consensus.cc:2272] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.833577 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet ae2927be5f3a440286ee0fe5ce818018 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.833945 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet ae2927be5f3a440286ee0fe5ce818018 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.835098 8551 ts_tablet_manager.cc:1929] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.835171 8551 log.cc:1199] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/29396aa23c734dc78983623aca442ba6
I20251028 02:59:50.836076 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 29396aa23c734dc78983623aca442ba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.836206 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 5444c480b4c84863bfb85f6367227005 with delete_type TABLET_DATA_DELETED (replaced by tablet c5a698db1042424bad62bce674b2c44d) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.836973 8654 mvcc.cc:204] Tried to move back new op lower bound from 7215597120291786752 to 7215597120060014592. Current Snapshot: MvccSnapshot[applied={T|T < 7215597120291786752}]
I20251028 02:59:50.837494 8552 ts_tablet_manager.cc:1916] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.839098 8725 tablet.cc:1722] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:50.839603 8552 ts_tablet_manager.cc:1929] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.839669 8552 log.cc:1199] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/ae2927be5f3a440286ee0fe5ce818018
I20251028 02:59:50.839886 8552 ts_tablet_manager.cc:1950] T ae2927be5f3a440286ee0fe5ce818018 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.840144 8146 raft_consensus.cc:1275] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:50.840440 8552 tablet_replica.cc:333] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.840476 8285 raft_consensus.cc:1275] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:50.840566 8552 raft_consensus.cc:2243] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.840651 8552 raft_consensus.cc:2272] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.840595 8616 consensus_queue.cc:1048] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:50.842828 8713 consensus_queue.cc:1175] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20251028 02:59:50.843452 8550 ts_tablet_manager.cc:1916] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.843701 8545 tablet_bootstrap.cc:492] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.843783 8545 ts_tablet_manager.cc:1403] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20251028 02:59:50.844205 8549 consensus_queue.cc:1048] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:50.844305 8545 raft_consensus.cc:359] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.844398 8545 raft_consensus.cc:385] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.844431 8545 raft_consensus.cc:740] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.844553 8545 consensus_queue.cc:260] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.844869 8545 ts_tablet_manager.cc:1434] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.845064 8545 tablet_bootstrap.cc:492] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.845607 8550 ts_tablet_manager.cc:1929] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:50.845677 8550 log.cc:1199] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/29396aa23c734dc78983623aca442ba6
I20251028 02:59:50.845932 8550 ts_tablet_manager.cc:1950] T 29396aa23c734dc78983623aca442ba6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.846206 8545 tablet_bootstrap.cc:654] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.846649 8551 ts_tablet_manager.cc:1950] T 29396aa23c734dc78983623aca442ba6 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.846666 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 29396aa23c734dc78983623aca442ba6 (table default.Table_d656e1f10f1e4dba925e8578af15b760 [id=85b3a3370fda46dcb686d811f1b12172]) successfully deleted
I20251028 02:59:50.846943 8550 tablet_replica.cc:333] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.847070 8550 raft_consensus.cc:2243] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.847162 8550 raft_consensus.cc:2272] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.847360 8562 raft_consensus.cc:493] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.847450 8562 raft_consensus.cc:515] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.847669 8550 ts_tablet_manager.cc:1916] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.847836 8562 leader_election.cc:290] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:50.848404 8083 leader_election.cc:343] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: a3d108be7a734497a9619cd7c097b2e5
I20251028 02:59:50.848747 8551 tablet_replica.cc:333] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.848739 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3d108be7a734497a9619cd7c097b2e5" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:50.848903 8284 raft_consensus.cc:2468] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:50.849306 8083 leader_election.cc:304] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:50.849637 8562 raft_consensus.cc:2804] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.849722 8562 raft_consensus.cc:493] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.849828 8562 raft_consensus.cc:3060] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.850656 8562 raft_consensus.cc:515] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.851373 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3d108be7a734497a9619cd7c097b2e5" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:50.851482 8284 raft_consensus.cc:3060] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:50.851856 8083 leader_election.cc:343] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: a3d108be7a734497a9619cd7c097b2e5
I20251028 02:59:50.851946 8562 leader_election.cc:290] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.849642 8550 ts_tablet_manager.cc:1929] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.852073 8550 log.cc:1199] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ae2927be5f3a440286ee0fe5ce818018
I20251028 02:59:50.852319 8284 raft_consensus.cc:2468] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:50.852624 8550 ts_tablet_manager.cc:1950] T ae2927be5f3a440286ee0fe5ce818018 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.853374 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet ae2927be5f3a440286ee0fe5ce818018 (table default.Table_ff1533cf65f64af7919ffc1aeadd1cac [id=f490a46696124e7ea69612e94d7321d2]) successfully deleted
I20251028 02:59:50.854171 8083 leader_election.cc:304] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:50.854452 8726 tablet.cc:1722] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:50.855654 8562 raft_consensus.cc:2804] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.855867 8562 raft_consensus.cc:697] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:50.856045 8551 raft_consensus.cc:2243] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.856176 8551 raft_consensus.cc:2272] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.856288 8562 consensus_queue.cc:237] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.856616 8552 ts_tablet_manager.cc:1916] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.859664 8753 tablet.cc:1722] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:50.859799 8551 ts_tablet_manager.cc:1916] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.859870 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 29396aa23c734dc78983623aca442ba6 (table default.Table_d656e1f10f1e4dba925e8578af15b760 [id=85b3a3370fda46dcb686d811f1b12172]) successfully deleted
I20251028 02:59:50.860674 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet ae2927be5f3a440286ee0fe5ce818018 (table default.Table_ff1533cf65f64af7919ffc1aeadd1cac [id=f490a46696124e7ea69612e94d7321d2]) successfully deleted
I20251028 02:59:50.861443 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5
I20251028 02:59:50.861716 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 deleted and replaced by tablet 8643565320504fe6b6154e936e55fca9
I20251028 02:59:50.861946 8725 tablet.cc:1722] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:50.861936 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_966ea54a4f6c408cb6386c3f729e9a9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.862346 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_966ea54a4f6c408cb6386c3f729e9a9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.863191 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet caff8b9ed8a54ccbbcdfac2d62bc2a71
I20251028 02:59:50.863478 7969 catalog_manager.cc:6719] ReplaceTablet: tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 deleted and replaced by tablet 773b2414ba1c46d7b6982768967d307d
I20251028 02:59:50.865087 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 8643565320504fe6b6154e936e55fca9) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.865129 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 with delete_type TABLET_DATA_DELETED (replaced by tablet 773b2414ba1c46d7b6982768967d307d) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.865314 8550 tablet_replica.cc:333] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.865572 8550 raft_consensus.cc:2243] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.865757 8550 raft_consensus.cc:2272] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.866427 8550 ts_tablet_manager.cc:1916] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.868640 8552 ts_tablet_manager.cc:1929] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.868721 8552 log.cc:1199] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/29396aa23c734dc78983623aca442ba6
I20251028 02:59:50.868939 8552 ts_tablet_manager.cc:1950] T 29396aa23c734dc78983623aca442ba6 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.869264 8648 raft_consensus.cc:493] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.869359 8648 raft_consensus.cc:515] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.869524 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9b0abfe5364749bd86b9916480828ffd
I20251028 02:59:50.869750 8648 leader_election.cc:290] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:50.869936 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 8643565320504fe6b6154e936e55fca9) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.870029 8550 ts_tablet_manager.cc:1929] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.870170 8550 log.cc:1199] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/caff8b9ed8a54ccbbcdfac2d62bc2a71
I20251028 02:59:50.870211 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0876f8b4ee634d9d92efe9613b3495be" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:50.870275 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b0abfe5364749bd86b9916480828ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.870298 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0876f8b4ee634d9d92efe9613b3495be" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:50.870329 8285 raft_consensus.cc:2468] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:50.870540 8550 ts_tablet_manager.cc:1950] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.870594 8356 leader_election.cc:304] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
W20251028 02:59:50.870667 8354 leader_election.cc:343] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:50.870744 8648 raft_consensus.cc:2804] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.870783 8648 raft_consensus.cc:493] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.870841 8648 raft_consensus.cc:3060] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.871502 8550 tablet_replica.cc:333] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.871668 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 (table default.Table_e33fdb2a96be46e5b88dad8b0c017acc [id=df4041614f8a466b96efff0231711ec1]) successfully deleted
I20251028 02:59:50.871737 8550 raft_consensus.cc:2243] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.871831 8648 raft_consensus.cc:515] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.871893 8550 raft_consensus.cc:2272] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.872180 8648 leader_election.cc:290] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:50.872545 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0876f8b4ee634d9d92efe9613b3495be" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:50.872599 8550 ts_tablet_manager.cc:1916] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.872618 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0876f8b4ee634d9d92efe9613b3495be" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:50.872713 8285 raft_consensus.cc:3060] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:50.872884 8354 leader_election.cc:343] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:50.873618 8285 raft_consensus.cc:2468] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:50.873905 8356 leader_election.cc:304] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:50.874119 8648 raft_consensus.cc:2804] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.874204 8648 raft_consensus.cc:697] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:50.874337 8648 consensus_queue.cc:237] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.875957 8550 ts_tablet_manager.cc:1929] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.876089 8550 log.cc:1199] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/0a3d0d8a0e5e4e34b97230c97b5f27b5
I20251028 02:59:50.876574 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 8643565320504fe6b6154e936e55fca9) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.876626 8550 ts_tablet_manager.cc:1950] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.877589 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 with delete_type TABLET_DATA_DELETED (replaced by tablet 773b2414ba1c46d7b6982768967d307d) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.877888 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 (table default.Table_15bfc391c65b4379b26b45ef36d49a30 [id=a9662d8045574f0abf81c9e6bc2f180c]) successfully deleted
W20251028 02:59:50.878079 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9b0abfe5364749bd86b9916480828ffd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b0abfe5364749bd86b9916480828ffd already in progress: creating tablet
I20251028 02:59:50.878140 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 with delete_type TABLET_DATA_DELETED (replaced by tablet 773b2414ba1c46d7b6982768967d307d) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.878320 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 29396aa23c734dc78983623aca442ba6 (table default.Table_d656e1f10f1e4dba925e8578af15b760 [id=85b3a3370fda46dcb686d811f1b12172]) successfully deleted
I20251028 02:59:50.878369 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1023c48e40024553b0795a912f2fdd18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.878739 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b0abfe5364749bd86b9916480828ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:50.878815 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1023c48e40024553b0795a912f2fdd18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.878888 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b0abfe5364749bd86b9916480828ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.878381 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_809e808b3698494faa7ad0258f9d5774"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.879773 8551 ts_tablet_manager.cc:1929] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.879849 8551 log.cc:1199] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/5444c480b4c84863bfb85f6367227005
W20251028 02:59:50.879956 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_809e808b3698494faa7ad0258f9d5774 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.880064 8551 ts_tablet_manager.cc:1950] T 5444c480b4c84863bfb85f6367227005 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.880748 8552 tablet_replica.cc:333] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.880720 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 5444c480b4c84863bfb85f6367227005 (table default.Table_c1124b12dc1640bcbb89fa3d6c624c75 [id=eb9bbde4c5c54a928e854dab13c17ee5]) successfully deleted
I20251028 02:59:50.880903 8552 raft_consensus.cc:2243] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.881004 8552 raft_consensus.cc:2272] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.881480 8552 ts_tablet_manager.cc:1916] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.881842 8648 raft_consensus.cc:493] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.881918 8648 raft_consensus.cc:515] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.882292 8648 leader_election.cc:290] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:50.882606 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da5a607a1e7f4ee9b7468b7a12b5a484" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:50.882689 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da5a607a1e7f4ee9b7468b7a12b5a484" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:50.882742 8146 raft_consensus.cc:2468] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:50.882807 8285 raft_consensus.cc:2468] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:50.883066 8354 leader_election.cc:304] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.882573 8551 tablet_replica.cc:333] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.883350 8551 raft_consensus.cc:2243] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.883519 8551 raft_consensus.cc:2272] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.885488 8552 ts_tablet_manager.cc:1929] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.883579 8648 raft_consensus.cc:2804] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.885634 8648 raft_consensus.cc:493] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.885702 8648 raft_consensus.cc:3060] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.886601 8648 raft_consensus.cc:515] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.887290 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da5a607a1e7f4ee9b7468b7a12b5a484" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:50.887425 8146 raft_consensus.cc:3060] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.888335 8146 raft_consensus.cc:2468] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:50.888659 8354 leader_election.cc:304] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:50.888718 8648 leader_election.cc:290] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:50.888840 8616 raft_consensus.cc:2804] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.889091 8616 raft_consensus.cc:697] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:50.889250 8616 consensus_queue.cc:237] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.889446 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c4b5b1d7c738413f80b8aed09b597c60
I20251028 02:59:50.889581 8551 ts_tablet_manager.cc:1916] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.889740 7960 catalog_manager.cc:6719] ReplaceTablet: tablet c4b5b1d7c738413f80b8aed09b597c60 deleted and replaced by tablet 3e7dbafe267a4c62b0418d1b26585aad
I20251028 02:59:50.891826 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet c4b5b1d7c738413f80b8aed09b597c60 with delete_type TABLET_DATA_DELETED (replaced by tablet 3e7dbafe267a4c62b0418d1b26585aad) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.892947 8551 ts_tablet_manager.cc:1929] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.893023 8551 log.cc:1199] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ae2927be5f3a440286ee0fe5ce818018
I20251028 02:59:50.890631 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da5a607a1e7f4ee9b7468b7a12b5a484" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:50.893369 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet c4b5b1d7c738413f80b8aed09b597c60 with delete_type TABLET_DATA_DELETED (replaced by tablet 3e7dbafe267a4c62b0418d1b26585aad) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.893419 8285 raft_consensus.cc:3060] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.893667 8550 tablet_replica.cc:333] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.893815 8550 raft_consensus.cc:2243] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:50.893976 8550 pending_rounds.cc:70] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:50.894008 8550 pending_rounds.cc:77] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597120826150912 op_type: NO_OP noop_request { }
I20251028 02:59:50.894064 8550 raft_consensus.cc:2889] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:50.894114 8550 raft_consensus.cc:2272] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.894284 8285 raft_consensus.cc:2468] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:50.893287 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet c4b5b1d7c738413f80b8aed09b597c60 with delete_type TABLET_DATA_DELETED (replaced by tablet 3e7dbafe267a4c62b0418d1b26585aad) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.895057 8552 log.cc:1199] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/5444c480b4c84863bfb85f6367227005
I20251028 02:59:50.895260 8753 tablet.cc:1722] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:50.895361 8552 ts_tablet_manager.cc:1950] T 5444c480b4c84863bfb85f6367227005 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 02:59:50.895520 8747 consensus_queue.cc:1175] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20251028 02:59:50.896195 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 5444c480b4c84863bfb85f6367227005 (table default.Table_c1124b12dc1640bcbb89fa3d6c624c75 [id=eb9bbde4c5c54a928e854dab13c17ee5]) successfully deleted
I20251028 02:59:50.896376 8550 ts_tablet_manager.cc:1916] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.896494 8545 tablet_bootstrap.cc:492] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.896580 8545 ts_tablet_manager.cc:1403] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.052s user 0.001s sys 0.000s
I20251028 02:59:50.897070 8545 raft_consensus.cc:359] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.897183 8545 raft_consensus.cc:385] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.897226 8545 raft_consensus.cc:740] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.897156 8552 tablet_replica.cc:333] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.896620 8551 ts_tablet_manager.cc:1950] T ae2927be5f3a440286ee0fe5ce818018 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.897503 8552 raft_consensus.cc:2243] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.897342 8545 consensus_queue.cc:260] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.897810 8545 ts_tablet_manager.cc:1434] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.897982 8545 tablet_bootstrap.cc:492] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.898487 8550 ts_tablet_manager.cc:1929] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:50.898551 8550 log.cc:1199] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c4b5b1d7c738413f80b8aed09b597c60
I20251028 02:59:50.898808 8550 ts_tablet_manager.cc:1950] T c4b5b1d7c738413f80b8aed09b597c60 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.898937 8545 tablet_bootstrap.cc:654] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.900007 8552 raft_consensus.cc:2272] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.901013 8544 tablet_bootstrap.cc:492] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.901095 8544 ts_tablet_manager.cc:1403] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.174s user 0.001s sys 0.001s
I20251028 02:59:50.901572 8544 raft_consensus.cc:359] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.901687 8544 raft_consensus.cc:385] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.901726 8544 raft_consensus.cc:740] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.901774 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ae2927be5f3a440286ee0fe5ce818018 (table default.Table_ff1533cf65f64af7919ffc1aeadd1cac [id=f490a46696124e7ea69612e94d7321d2]) successfully deleted
I20251028 02:59:50.901845 8544 consensus_queue.cc:260] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.902005 8551 tablet_replica.cc:333] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.902124 8551 raft_consensus.cc:2243] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:50.902141 8544 ts_tablet_manager.cc:1434] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.902320 8544 tablet_bootstrap.cc:492] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.903071 8551 pending_rounds.cc:70] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:50.903158 8551 pending_rounds.cc:77] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597120429154304 op_type: NO_OP noop_request { }
I20251028 02:59:50.903225 8551 raft_consensus.cc:2889] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:50.903290 8551 raft_consensus.cc:2272] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.903905 8544 tablet_bootstrap.cc:654] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.904853 8726 tablet.cc:1717] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Skipping requested alter to schema version 3, tablet already version 3
I20251028 02:59:50.905417 8725 tablet.cc:1717] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: Skipping requested alter to schema version 3, tablet already version 3
I20251028 02:59:50.906075 8553 raft_consensus.cc:493] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.906170 8553 raft_consensus.cc:515] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.906322 8753 tablet.cc:1717] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: Skipping requested alter to schema version 3, tablet already version 3
I20251028 02:59:50.906571 8553 leader_election.cc:290] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.906807 8546 tablet_bootstrap.cc:492] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.906900 8546 ts_tablet_manager.cc:1403] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.085s user 0.002s sys 0.000s
I20251028 02:59:50.907377 8545 tablet_bootstrap.cc:492] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.907461 8545 ts_tablet_manager.cc:1403] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:50.907418 8546 raft_consensus.cc:359] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.907514 8546 raft_consensus.cc:385] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.907539 8546 raft_consensus.cc:740] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.907651 8546 consensus_queue.cc:260] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.907923 8546 ts_tablet_manager.cc:1434] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.908099 8546 tablet_bootstrap.cc:492] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.908092 8545 raft_consensus.cc:359] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.908188 8545 raft_consensus.cc:385] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.908218 8545 raft_consensus.cc:740] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
W20251028 02:59:50.908840 8218 leader_election.cc:343] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: f74e7c0e7d634679bb70cf3515aafe22
I20251028 02:59:50.909111 8546 tablet_bootstrap.cc:654] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.909391 8545 consensus_queue.cc:260] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:50.909955 8220 leader_election.cc:343] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: f74e7c0e7d634679bb70cf3515aafe22
I20251028 02:59:50.910029 8220 leader_election.cc:304] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:50.910357 8553 raft_consensus.cc:2749] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:50.911721 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet c4b5b1d7c738413f80b8aed09b597c60 (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:50.914865 7962 catalog_manager.cc:5649] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
W20251028 02:59:50.915617 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 206b71b0f10c42fd81f291a27fffc8a2 consensus state has no leader
I20251028 02:59:50.916154 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 102fc67bcf7f4336a26220fdb4f7a4d9 (DEFAULT_TABLE table=default.Table_49d0ca6e0aa54b309562cc456bcf1be5 [id=7f26fae434ec4bdda077c8ceb5350aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.916447 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102fc67bcf7f4336a26220fdb4f7a4d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.917536 8544 tablet_bootstrap.cc:492] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:50.917618 8544 ts_tablet_manager.cc:1403] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20251028 02:59:50.918076 8544 raft_consensus.cc:359] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.918174 8544 raft_consensus.cc:385] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.918205 8544 raft_consensus.cc:740] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.918345 8544 consensus_queue.cc:260] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.918620 8544 ts_tablet_manager.cc:1434] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:50.918732 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 102fc67bcf7f4336a26220fdb4f7a4d9 (DEFAULT_TABLE table=default.Table_49d0ca6e0aa54b309562cc456bcf1be5 [id=7f26fae434ec4bdda077c8ceb5350aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.918809 8544 tablet_bootstrap.cc:492] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:50.919008 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102fc67bcf7f4336a26220fdb4f7a4d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.919039 8333 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:50.919155 8333 heartbeater.cc:499] Master 127.7.154.189:45473 was elected leader, sending a full tablet report...
I20251028 02:59:50.920490 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 482d7541b350426f9ff16448a24daeed (DEFAULT_TABLE table=default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.920625 8544 tablet_bootstrap.cc:654] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.920712 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 482d7541b350426f9ff16448a24daeed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.921770 8661 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ccbdf478f18948d19de96574a685bfda" instance_seqno: 1761620389479519) as {username='slave'} at 127.7.154.130:37319; Asking this server to re-register.
I20251028 02:59:50.922587 8333 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:50.922844 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 482d7541b350426f9ff16448a24daeed (DEFAULT_TABLE table=default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.923079 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 482d7541b350426f9ff16448a24daeed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.924376 8196 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:50.924513 8196 heartbeater.cc:499] Master 127.7.154.189:45473 was elected leader, sending a full tablet report...
I20251028 02:59:50.924607 8671 raft_consensus.cc:3060] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 1 FOLLOWER]: Advancing to term 2
I20251028 02:59:50.926405 8123 tablet_service.cc:1505] Processing CreateTablet for tablet cef31f29cc1a4aaa83624aac96f3da04 (DEFAULT_TABLE table=default.Table_dbd38ab02bf5402aadd9c1bc0dfb7ba2 [id=960dc85e288d4ae6936f98938e123cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.926673 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef31f29cc1a4aaa83624aac96f3da04. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.928972 8546 tablet_bootstrap.cc:492] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.929072 8546 ts_tablet_manager.cc:1403] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20251028 02:59:50.929622 8546 raft_consensus.cc:359] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.929728 8546 raft_consensus.cc:385] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.929785 8546 raft_consensus.cc:740] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.929919 8546 consensus_queue.cc:260] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.930011 8545 ts_tablet_manager.cc:1434] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.023s user 0.001s sys 0.000s
I20251028 02:59:50.930207 8546 ts_tablet_manager.cc:1434] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.930375 8545 tablet_bootstrap.cc:492] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:50.930402 8546 tablet_bootstrap.cc:492] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.931350 8545 tablet_bootstrap.cc:654] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.931656 8546 tablet_bootstrap.cc:654] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.911391 8551 ts_tablet_manager.cc:1916] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.915414 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 102fc67bcf7f4336a26220fdb4f7a4d9 (DEFAULT_TABLE table=default.Table_49d0ca6e0aa54b309562cc456bcf1be5 [id=7f26fae434ec4bdda077c8ceb5350aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.932394 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102fc67bcf7f4336a26220fdb4f7a4d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.933616 8398 tablet_service.cc:1505] Processing CreateTablet for tablet cef31f29cc1a4aaa83624aac96f3da04 (DEFAULT_TABLE table=default.Table_dbd38ab02bf5402aadd9c1bc0dfb7ba2 [id=960dc85e288d4ae6936f98938e123cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.933856 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef31f29cc1a4aaa83624aac96f3da04. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.916366 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 482d7541b350426f9ff16448a24daeed (DEFAULT_TABLE table=default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.934461 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 482d7541b350426f9ff16448a24daeed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.918548 8469 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:50.936009 8469 heartbeater.cc:499] Master 127.7.154.189:45473 was elected leader, sending a full tablet report...
I20251028 02:59:50.937665 8260 tablet_service.cc:1505] Processing CreateTablet for tablet cef31f29cc1a4aaa83624aac96f3da04 (DEFAULT_TABLE table=default.Table_dbd38ab02bf5402aadd9c1bc0dfb7ba2 [id=960dc85e288d4ae6936f98938e123cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:50.937906 8546 tablet_bootstrap.cc:492] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.937944 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef31f29cc1a4aaa83624aac96f3da04. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:50.937999 8546 ts_tablet_manager.cc:1403] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:50.938482 8546 raft_consensus.cc:359] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.938580 8546 raft_consensus.cc:385] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.938608 8546 raft_consensus.cc:740] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.938769 8546 consensus_queue.cc:260] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.939163 8546 ts_tablet_manager.cc:1434] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.939363 8546 tablet_bootstrap.cc:492] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.940215 8546 tablet_bootstrap.cc:654] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.942889 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6d18f1aded5b45cd85d34e4790ad8d40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.943372 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d18f1aded5b45cd85d34e4790ad8d40 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.943881 8551 ts_tablet_manager.cc:1929] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:50.943953 8551 log.cc:1199] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/0a3d0d8a0e5e4e34b97230c97b5f27b5
I20251028 02:59:50.944206 8551 ts_tablet_manager.cc:1950] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:50.945811 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_597d948d86e446488fce9852ca89032d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.946180 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_597d948d86e446488fce9852ca89032d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.946424 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bb2a8beb93a74d8eb05afe4fb154bbfa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.946722 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb2a8beb93a74d8eb05afe4fb154bbfa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.947582 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 (table default.Table_15bfc391c65b4379b26b45ef36d49a30 [id=a9662d8045574f0abf81c9e6bc2f180c]) successfully deleted
I20251028 02:59:50.947751 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 0876f8b4ee634d9d92efe9613b3495be (table default.Table_b026ff28518f49aea62b20dd70917a7a [id=7a2a6ab3c9744942a9413c6cb057f527]). Expected version 1 got 0
I20251028 02:59:50.948324 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d4ddd31975814e4b9c829a132cdc6dad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.948629 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4ddd31975814e4b9c829a132cdc6dad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.949278 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2386ea3c08fe48c381c7fc2321ecbc86
I20251028 02:59:50.949637 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet da5a607a1e7f4ee9b7468b7a12b5a484
I20251028 02:59:50.952273 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 2386ea3c08fe48c381c7fc2321ecbc86 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.952769 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet da5a607a1e7f4ee9b7468b7a12b5a484 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.953218 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 5444c480b4c84863bfb85f6367227005 with delete_type TABLET_DATA_DELETED (replaced by tablet c5a698db1042424bad62bce674b2c44d) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.953756 8545 tablet_bootstrap.cc:492] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:50.953858 8545 ts_tablet_manager.cc:1403] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.002s
I20251028 02:59:50.954339 8545 raft_consensus.cc:359] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.954452 8545 raft_consensus.cc:385] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.954483 8545 raft_consensus.cc:740] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.955034 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet ae2927be5f3a440286ee0fe5ce818018 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.955210 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 5444c480b4c84863bfb85f6367227005 with delete_type TABLET_DATA_DELETED (replaced by tablet c5a698db1042424bad62bce674b2c44d) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.955237 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 2386ea3c08fe48c381c7fc2321ecbc86 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.955396 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 29396aa23c734dc78983623aca442ba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.955488 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet a3d108be7a734497a9619cd7c097b2e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.955693 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet a3d108be7a734497a9619cd7c097b2e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.956398 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet f74e7c0e7d634679bb70cf3515aafe22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:50.956820 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet ae2927be5f3a440286ee0fe5ce818018 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.957181 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 8643565320504fe6b6154e936e55fca9) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.958613 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 2386ea3c08fe48c381c7fc2321ecbc86 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.959018 7962 catalog_manager.cc:5649] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:50.959144 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c320b509540344b7895fb967b844a408"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.959255 7962 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 0876f8b4ee634d9d92efe9613b3495be (table default.Table_b026ff28518f49aea62b20dd70917a7a [id=7a2a6ab3c9744942a9413c6cb057f527]). Expected version 1 got 0
I20251028 02:59:50.959240 8545 consensus_queue.cc:260] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:50.959542 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c320b509540344b7895fb967b844a408 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.960261 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e33fdb2a96be46e5b88dad8b0c017acc" } new_table_name: "default.Table_b44dd3c16841435a8dd467a1da28db7a" modify_external_catalogs: true
I20251028 02:59:50.960868 8550 tablet_replica.cc:333] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.961017 8550 raft_consensus.cc:2243] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.960990 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_50b58157f999483da285fc8730aba0e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:50.961153 8550 raft_consensus.cc:2272] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.961262 8545 ts_tablet_manager.cc:1434] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
W20251028 02:59:50.961319 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50b58157f999483da285fc8730aba0e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.961798 8550 ts_tablet_manager.cc:1916] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.963407 8545 tablet_bootstrap.cc:492] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:50.964375 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ae2927be5f3a440286ee0fe5ce818018 because the tablet was not found. No further retry: Not found: Tablet not found: ae2927be5f3a440286ee0fe5ce818018
W20251028 02:59:50.964553 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 5444c480b4c84863bfb85f6367227005 because the tablet was not found. No further retry: Not found: Tablet not found: 5444c480b4c84863bfb85f6367227005
I20251028 02:59:50.964659 8545 tablet_bootstrap.cc:654] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.964713 8550 ts_tablet_manager.cc:1929] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:50.964777 8550 log.cc:1199] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2386ea3c08fe48c381c7fc2321ecbc86
I20251028 02:59:50.964928 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet da5a607a1e7f4ee9b7468b7a12b5a484 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:50.965027 8550 ts_tablet_manager.cc:1950] T 2386ea3c08fe48c381c7fc2321ecbc86 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.965160 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet da5a607a1e7f4ee9b7468b7a12b5a484 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:50.965559 8552 ts_tablet_manager.cc:1916] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.965726 8550 tablet_replica.cc:333] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:50.965831 8550 raft_consensus.cc:2243] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:50.965974 8550 pending_rounds.cc:70] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:50.966007 8550 pending_rounds.cc:77] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597120948936704 op_type: NO_OP noop_request { }
I20251028 02:59:50.966255 8420 raft_consensus.cc:1275] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:50.966804 8562 consensus_queue.cc:1048] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:50.967063 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2386ea3c08fe48c381c7fc2321ecbc86 (table default.Table_5036a52b716149b2a336ce9c3a019ed1 [id=9d19f4b99b02446c8bbc1263a9b70c4e]) successfully deleted
W20251028 02:59:50.967223 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 29396aa23c734dc78983623aca442ba6 because the tablet was not found. No further retry: Not found: Tablet not found: 29396aa23c734dc78983623aca442ba6
I20251028 02:59:50.967686 8552 ts_tablet_manager.cc:1929] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:50.967747 8285 raft_consensus.cc:1275] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:50.968176 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_af0da4b12fef412fa59f7ec5fa3fc660"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:50.968561 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af0da4b12fef412fa59f7ec5fa3fc660 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:50.969185 8562 consensus_queue.cc:1048] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:50.969854 8546 tablet_bootstrap.cc:492] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:50.969877 8146 raft_consensus.cc:1275] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:50.970101 8553 raft_consensus.cc:493] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:50.970175 8553 raft_consensus.cc:515] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.967763 8552 log.cc:1199] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/0a3d0d8a0e5e4e34b97230c97b5f27b5
I20251028 02:59:50.970528 8553 leader_election.cc:290] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.970650 8552 ts_tablet_manager.cc:1950] T 0a3d0d8a0e5e4e34b97230c97b5f27b5 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:50.971282 8552 tablet_replica.cc:333] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:50.971436 8552 raft_consensus.cc:2243] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:50.971521 8552 raft_consensus.cc:2272] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.971657 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 (table default.Table_15bfc391c65b4379b26b45ef36d49a30 [id=a9662d8045574f0abf81c9e6bc2f180c]) successfully deleted
I20251028 02:59:50.970641 8546 ts_tablet_manager.cc:1403] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20251028 02:59:50.971999 8419 raft_consensus.cc:1275] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:50.972208 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70e625899ec546f7add9948e59d289af" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:50.972203 8546 raft_consensus.cc:359] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:50.972288 8546 raft_consensus.cc:385] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:50.972301 8421 raft_consensus.cc:2468] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:50.972316 8546 raft_consensus.cc:740] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:50.972404 8546 consensus_queue.cc:260] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:50.972697 8546 ts_tablet_manager.cc:1434] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:50.972851 8546 tablet_bootstrap.cc:492] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:50.973197 8220 leader_election.cc:304] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:50.973352 8631 raft_consensus.cc:2804] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:50.973331 8553 consensus_queue.cc:1048] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:50.973393 8631 raft_consensus.cc:493] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:50.973426 8631 raft_consensus.cc:3060] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.974387 8631 raft_consensus.cc:515] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.974726 8631 leader_election.cc:290] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:50.966074 8550 raft_consensus.cc:2889] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:50.975230 8550 raft_consensus.cc:2272] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.975225 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70e625899ec546f7add9948e59d289af" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:50.975349 8419 raft_consensus.cc:3060] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:50.975431 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70e625899ec546f7add9948e59d289af" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:50.976056 8218 leader_election.cc:343] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:50.976346 8419 raft_consensus.cc:2468] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:50.976629 8550 ts_tablet_manager.cc:1916] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.976660 8220 leader_election.cc:304] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:50.976830 8553 consensus_queue.cc:1048] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:50.977094 8553 raft_consensus.cc:2804] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:50.977164 8553 raft_consensus.cc:697] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:50.977274 8553 consensus_queue.cc:237] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:50.980119 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "70e625899ec546f7add9948e59d289af" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:50.980625 8218 leader_election.cc:343] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:50.985244 8546 tablet_bootstrap.cc:654] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:50.985430 8550 ts_tablet_manager.cc:1929] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:50.985507 8550 log.cc:1199] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a3d108be7a734497a9619cd7c097b2e5
I20251028 02:59:50.985745 8550 ts_tablet_manager.cc:1950] T a3d108be7a734497a9619cd7c097b2e5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:50.985992 8551 tablet_replica.cc:333] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:50.986138 8551 raft_consensus.cc:2243] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:50.986301 8551 pending_rounds.cc:70] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:50.986336 8551 pending_rounds.cc:77] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597120843444224 op_type: NO_OP noop_request { }
I20251028 02:59:50.986394 8551 raft_consensus.cc:2889] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:50.986449 8551 raft_consensus.cc:2272] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:50.989692 8551 ts_tablet_manager.cc:1916] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:50.991816 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 773b2414ba1c46d7b6982768967d307d has no consensus state
I20251028 02:59:50.993474 8552 ts_tablet_manager.cc:1916] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:50.994117 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 0876f8b4ee634d9d92efe9613b3495be (table default.Table_b026ff28518f49aea62b20dd70917a7a [id=7a2a6ab3c9744942a9413c6cb057f527]). Expected version 1 got 0
I20251028 02:59:50.994292 8725 tablet.cc:1722] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:50.995440 7965 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_1cdf4b1794504307906ac65e49eb2c71 [id=b34ab58e43e943909c9bdd79ad69e639] alter complete (version 3)
I20251028 02:59:50.996990 8726 tablet.cc:1722] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:50.998911 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a3d108be7a734497a9619cd7c097b2e5 (table default.Table_ddb39e36d68643c2909826b661af4128 [id=8f89769a43034f7d9f783602532e7fef]) successfully deleted
I20251028 02:59:50.999186 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 2386ea3c08fe48c381c7fc2321ecbc86 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.001247 8550 tablet_replica.cc:333] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.001377 8550 raft_consensus.cc:2243] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.001487 8550 raft_consensus.cc:2272] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.002409 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a7f1f8afc727489faff5840c8881f773" } new_table_name: "default.Table_1f3665f140a54beebcb5f655ce120846" modify_external_catalogs: true
I20251028 02:59:51.003724 8552 ts_tablet_manager.cc:1929] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.003841 8552 log.cc:1199] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/caff8b9ed8a54ccbbcdfac2d62bc2a71
I20251028 02:59:51.003816 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_dbd38ab02bf5402aadd9c1bc0dfb7ba2" } new_table_name: "default.Table_6532259aa4cc47ee85ab998f23912c02" modify_external_catalogs: true
I20251028 02:59:51.004083 8552 ts_tablet_manager.cc:1950] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.004803 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 (table default.Table_b44dd3c16841435a8dd467a1da28db7a [id=df4041614f8a466b96efff0231711ec1]) successfully deleted
I20251028 02:59:51.005122 8552 tablet_replica.cc:333] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.005255 8552 raft_consensus.cc:2243] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.005363 8552 raft_consensus.cc:2272] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.005796 8669 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" instance_seqno: 1761620389287366) as {username='slave'} at 127.7.154.129:58593; Asking this server to re-register.
I20251028 02:59:51.006268 8727 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9343ca8510de4eaa9945786e054a866d. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:51.006393 8727 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:51.006490 8727 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:51.006549 8727 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:51.005964 8550 ts_tablet_manager.cc:1916] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.006987 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4beb9b222fd14525832fb1ff5029e701" } new_table_name: "default.Table_596fa65222fb43afbb4d4b620ee0a0c1" modify_external_catalogs: true
I20251028 02:59:51.007179 8668 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" instance_seqno: 1761620389689963) as {username='slave'} at 127.7.154.131:44063; Asking this server to re-register.
I20251028 02:59:51.007824 8544 tablet_bootstrap.cc:492] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.007921 8544 ts_tablet_manager.cc:1403] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.089s user 0.000s sys 0.002s
I20251028 02:59:51.008458 8544 raft_consensus.cc:359] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.008569 8544 raft_consensus.cc:385] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.008601 8544 raft_consensus.cc:740] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.008715 8544 consensus_queue.cc:260] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.008841 8550 ts_tablet_manager.cc:1929] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.008909 8550 log.cc:1199] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/da5a607a1e7f4ee9b7468b7a12b5a484
I20251028 02:59:51.008972 8544 ts_tablet_manager.cc:1434] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251028 02:59:51.009125 8550 ts_tablet_manager.cc:1950] T da5a607a1e7f4ee9b7468b7a12b5a484 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.009159 8544 tablet_bootstrap.cc:492] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.009788 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet da5a607a1e7f4ee9b7468b7a12b5a484 (table default.Table_bf33aa6089da40008abddb5fe2796251 [id=dfeb1ac7ead5405c811d04273658497b]) successfully deleted
I20251028 02:59:51.010177 8544 tablet_bootstrap.cc:654] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:51.011225 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 2386ea3c08fe48c381c7fc2321ecbc86 because the tablet was not found. No further retry: Not found: Tablet not found: 2386ea3c08fe48c381c7fc2321ecbc86
I20251028 02:59:51.012286 8196 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:51.012428 8648 raft_consensus.cc:493] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.012514 8648 raft_consensus.cc:515] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.012717 8726 tablet.cc:1722] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.012895 8648 leader_election.cc:290] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:51.013295 8469 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:51.013288 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "36d7cf962de845cfaca691be18a1bd63" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.013423 8285 raft_consensus.cc:2468] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.013744 8356 leader_election.cc:304] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:51.014884 8670 ts_manager.cc:194] Registered new tserver with Master: ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.017172 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "36d7cf962de845cfaca691be18a1bd63" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.017215 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 206b71b0f10c42fd81f291a27fffc8a2 (table default.Table_43b9d6e2a6af4b25ae6e80ecccbe4392 [id=d0c0560068b54a5e98d13b2196c75aa7]). Expected version 1 got 0
I20251028 02:59:51.017329 7965 catalog_manager.cc:5649] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
W20251028 02:59:51.017624 8354 leader_election.cc:343] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.018105 8725 tablet.cc:1722] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.018210 8668 ts_manager.cc:194] Registered new tserver with Master: d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.018451 8648 raft_consensus.cc:2804] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.018513 8648 raft_consensus.cc:493] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.018553 8648 raft_consensus.cc:3060] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.018517 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 3e7dbafe267a4c62b0418d1b26585aad (DEFAULT_TABLE table=default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.018815 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e7dbafe267a4c62b0418d1b26585aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.019667 8648 raft_consensus.cc:515] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.020071 8648 leader_election.cc:290] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:51.020605 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "36d7cf962de845cfaca691be18a1bd63" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:51.020946 8354 leader_election.cc:343] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.021106 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "36d7cf962de845cfaca691be18a1bd63" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.021216 8285 raft_consensus.cc:3060] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.021381 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 4b4f077b41fc4f8b90db7392984b48a8 (DEFAULT_TABLE table=default.Table_966ea54a4f6c408cb6386c3f729e9a9a [id=a6ae2be5375e4328848f28f43dfc49ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.021602 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b4f077b41fc4f8b90db7392984b48a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.022296 8285 raft_consensus.cc:2468] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.022424 8670 ts_manager.cc:194] Registered new tserver with Master: fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:51.022734 8356 leader_election.cc:304] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.023530 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 4b4f077b41fc4f8b90db7392984b48a8 (DEFAULT_TABLE table=default.Table_966ea54a4f6c408cb6386c3f729e9a9a [id=a6ae2be5375e4328848f28f43dfc49ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.023804 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b4f077b41fc4f8b90db7392984b48a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.026885 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 4b4f077b41fc4f8b90db7392984b48a8 (DEFAULT_TABLE table=default.Table_966ea54a4f6c408cb6386c3f729e9a9a [id=a6ae2be5375e4328848f28f43dfc49ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.027115 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 990a32edd54d4d85bcb6108cfb5be4d7 (DEFAULT_TABLE table=default.Table_809e808b3698494faa7ad0258f9d5774 [id=f1e543817e8b48c8bf0e7f5ddb57330c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.027187 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b4f077b41fc4f8b90db7392984b48a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.027374 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 990a32edd54d4d85bcb6108cfb5be4d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.028505 8552 ts_tablet_manager.cc:1916] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.030542 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 773b2414ba1c46d7b6982768967d307d (DEFAULT_TABLE table=default.Table_b44dd3c16841435a8dd467a1da28db7a [id=df4041614f8a466b96efff0231711ec1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.030593 8552 ts_tablet_manager.cc:1929] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.030566 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 8643565320504fe6b6154e936e55fca9 (DEFAULT_TABLE table=default.Table_15bfc391c65b4379b26b45ef36d49a30 [id=a9662d8045574f0abf81c9e6bc2f180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.030652 8552 log.cc:1199] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9b0abfe5364749bd86b9916480828ffd
I20251028 02:59:51.030791 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773b2414ba1c46d7b6982768967d307d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.030807 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8643565320504fe6b6154e936e55fca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.030864 8552 ts_tablet_manager.cc:1950] T 9b0abfe5364749bd86b9916480828ffd P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.031925 8546 tablet_bootstrap.cc:492] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.032011 8546 ts_tablet_manager.cc:1403] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.000s
I20251028 02:59:51.032521 8546 raft_consensus.cc:359] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.032629 8546 raft_consensus.cc:385] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.032660 8546 raft_consensus.cc:740] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.032763 8265 tablet_service.cc:1505] Processing CreateTablet for tablet 3209160f7b0e4f1a8c8121725454d088 (DEFAULT_TABLE table=default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.032985 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3209160f7b0e4f1a8c8121725454d088. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.033196 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 8643565320504fe6b6154e936e55fca9 (DEFAULT_TABLE table=default.Table_15bfc391c65b4379b26b45ef36d49a30 [id=a9662d8045574f0abf81c9e6bc2f180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.033315 8546 consensus_queue.cc:260] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.033408 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8643565320504fe6b6154e936e55fca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.034888 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 990a32edd54d4d85bcb6108cfb5be4d7 (DEFAULT_TABLE table=default.Table_809e808b3698494faa7ad0258f9d5774 [id=f1e543817e8b48c8bf0e7f5ddb57330c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.035085 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 990a32edd54d4d85bcb6108cfb5be4d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.035851 8546 ts_tablet_manager.cc:1434] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:51.039419 8546 tablet_bootstrap.cc:492] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.040066 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 8643565320504fe6b6154e936e55fca9 (DEFAULT_TABLE table=default.Table_15bfc391c65b4379b26b45ef36d49a30 [id=a9662d8045574f0abf81c9e6bc2f180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.040334 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8643565320504fe6b6154e936e55fca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.040565 8546 tablet_bootstrap.cc:654] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:51.041409 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cef31f29cc1a4aaa83624aac96f3da04 consensus state has no leader
W20251028 02:59:51.041690 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 36d7cf962de845cfaca691be18a1bd63 consensus state has no leader
I20251028 02:59:51.042361 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 3209160f7b0e4f1a8c8121725454d088 (DEFAULT_TABLE table=default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.042515 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 3209160f7b0e4f1a8c8121725454d088 (DEFAULT_TABLE table=default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.042615 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3209160f7b0e4f1a8c8121725454d088. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.042733 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3209160f7b0e4f1a8c8121725454d088. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.043504 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 990a32edd54d4d85bcb6108cfb5be4d7 (DEFAULT_TABLE table=default.Table_809e808b3698494faa7ad0258f9d5774 [id=f1e543817e8b48c8bf0e7f5ddb57330c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.043740 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 990a32edd54d4d85bcb6108cfb5be4d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.046679 8122 tablet_service.cc:1505] Processing CreateTablet for tablet c5a698db1042424bad62bce674b2c44d (DEFAULT_TABLE table=default.Table_c1124b12dc1640bcbb89fa3d6c624c75 [id=eb9bbde4c5c54a928e854dab13c17ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.046924 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a698db1042424bad62bce674b2c44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.047276 8552 tablet_replica.cc:333] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.047454 8552 raft_consensus.cc:2243] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.047580 8552 raft_consensus.cc:2272] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.048370 8552 ts_tablet_manager.cc:1916] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.048393 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9b0abfe5364749bd86b9916480828ffd (table default.Table_ef5a7a00ec9e47349f4b15f619f609f7 [id=b73cb9b72fab4a68946bb2ce1ddcf026]) successfully deleted
I20251028 02:59:51.048815 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_43b9d6e2a6af4b25ae6e80ecccbe4392" } modify_external_catalogs: true
I20251028 02:59:51.048928 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_43b9d6e2a6af4b25ae6e80ecccbe4392" } modify_external_catalogs: true
I20251028 02:59:51.049172 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c1124b12dc1640bcbb89fa3d6c624c75" } modify_external_catalogs: true
I20251028 02:59:51.049299 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet 70e625899ec546f7add9948e59d289af from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.049311 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c1124b12dc1640bcbb89fa3d6c624c75" } modify_external_catalogs: true
I20251028 02:59:51.049228 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 85a0ee11b35545cca8db8754c0b6b7d7 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.051425 8549 raft_consensus.cc:2804] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.051528 8549 raft_consensus.cc:697] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:51.051676 8549 consensus_queue.cc:237] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.053076 8551 ts_tablet_manager.cc:1929] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:51.053153 8551 log.cc:1199] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/caff8b9ed8a54ccbbcdfac2d62bc2a71
I20251028 02:59:51.053396 8551 ts_tablet_manager.cc:1950] T caff8b9ed8a54ccbbcdfac2d62bc2a71 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.055039 8545 tablet_bootstrap.cc:492] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.055130 8545 ts_tablet_manager.cc:1403] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.092s user 0.002s sys 0.000s
I20251028 02:59:51.051486 8753 tablet.cc:1722] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.055656 8545 raft_consensus.cc:359] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.055783 8545 raft_consensus.cc:385] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.055819 8545 raft_consensus.cc:740] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.055929 8545 consensus_queue.cc:260] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.056205 8545 ts_tablet_manager.cc:1434] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.056391 8545 tablet_bootstrap.cc:492] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.057451 8545 tablet_bootstrap.cc:654] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.058115 8552 ts_tablet_manager.cc:1929] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.058194 8552 log.cc:1199] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c4b5b1d7c738413f80b8aed09b597c60
I20251028 02:59:51.058547 8749 tablet.cc:1722] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.058681 8265 tablet_service.cc:1505] Processing CreateTablet for tablet c5a698db1042424bad62bce674b2c44d (DEFAULT_TABLE table=default.Table_c1124b12dc1640bcbb89fa3d6c624c75 [id=eb9bbde4c5c54a928e854dab13c17ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.058945 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a698db1042424bad62bce674b2c44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.059399 8546 tablet_bootstrap.cc:492] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.059477 8546 ts_tablet_manager.cc:1403] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251028 02:59:51.059968 8546 raft_consensus.cc:359] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.060354 8546 raft_consensus.cc:385] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.060407 8546 raft_consensus.cc:740] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.060568 8546 consensus_queue.cc:260] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.062391 8551 tablet_replica.cc:333] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.064571 8546 ts_tablet_manager.cc:1434] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 02:59:51.065719 7962 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet d09229ca82eb49f2b4214b238c3ec097 (table default.Table_a7f1f8afc727489faff5840c8881f773 [id=7fe387bad5e740429e327d42f1efde22]). Expected version 1 got 0
I20251028 02:59:51.065860 7962 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 206b71b0f10c42fd81f291a27fffc8a2 (table default.Table_43b9d6e2a6af4b25ae6e80ecccbe4392 [id=d0c0560068b54a5e98d13b2196c75aa7]). Expected version 1 got 0
I20251028 02:59:51.065832 8397 tablet_service.cc:1505] Processing CreateTablet for tablet c5a698db1042424bad62bce674b2c44d (DEFAULT_TABLE table=default.Table_c1124b12dc1640bcbb89fa3d6c624c75 [id=eb9bbde4c5c54a928e854dab13c17ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.066107 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5a698db1042424bad62bce674b2c44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.067600 8546 tablet_bootstrap.cc:492] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.067641 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 773b2414ba1c46d7b6982768967d307d (DEFAULT_TABLE table=default.Table_b44dd3c16841435a8dd467a1da28db7a [id=df4041614f8a466b96efff0231711ec1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.067910 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773b2414ba1c46d7b6982768967d307d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.068418 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 773b2414ba1c46d7b6982768967d307d (DEFAULT_TABLE table=default.Table_b44dd3c16841435a8dd467a1da28db7a [id=df4041614f8a466b96efff0231711ec1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.068629 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773b2414ba1c46d7b6982768967d307d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.068646 8546 tablet_bootstrap.cc:654] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.069216 8552 ts_tablet_manager.cc:1950] T c4b5b1d7c738413f80b8aed09b597c60 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.069880 8552 tablet_replica.cc:333] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.069999 8552 raft_consensus.cc:2243] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.070322 8552 raft_consensus.cc:2272] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.071251 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 3e7dbafe267a4c62b0418d1b26585aad (DEFAULT_TABLE table=default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.071506 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e7dbafe267a4c62b0418d1b26585aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.071780 8539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:51.071882 8539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: This master's current role is: LEADER
I20251028 02:59:51.073202 8551 raft_consensus.cc:2243] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.073324 8551 raft_consensus.cc:2272] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.073720 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 3e7dbafe267a4c62b0418d1b26585aad (DEFAULT_TABLE table=default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.073977 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e7dbafe267a4c62b0418d1b26585aad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.075933 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c5a698db1042424bad62bce674b2c44d
I20251028 02:59:51.076285 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet caff8b9ed8a54ccbbcdfac2d62bc2a71 (table default.Table_b44dd3c16841435a8dd467a1da28db7a [id=df4041614f8a466b96efff0231711ec1]) successfully deleted
I20251028 02:59:51.076640 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c4b5b1d7c738413f80b8aed09b597c60 (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:51.076639 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet c5a698db1042424bad62bce674b2c44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:51.077078 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet c5a698db1042424bad62bce674b2c44d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5a698db1042424bad62bce674b2c44d already in progress: creating tablet
I20251028 02:59:51.077387 8551 ts_tablet_manager.cc:1916] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.077471 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet c5a698db1042424bad62bce674b2c44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.078526 8544 tablet_bootstrap.cc:492] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.078634 8544 ts_tablet_manager.cc:1403] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.070s user 0.000s sys 0.002s
I20251028 02:59:51.079452 8551 ts_tablet_manager.cc:1929] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.079525 8551 log.cc:1199] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9b0abfe5364749bd86b9916480828ffd
I20251028 02:59:51.079732 8551 ts_tablet_manager.cc:1950] T 9b0abfe5364749bd86b9916480828ffd P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.080345 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9b0abfe5364749bd86b9916480828ffd (table default.Table_ef5a7a00ec9e47349f4b15f619f609f7 [id=b73cb9b72fab4a68946bb2ce1ddcf026]) successfully deleted
I20251028 02:59:51.079191 8544 raft_consensus.cc:359] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.080935 8544 raft_consensus.cc:385] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.080982 8544 raft_consensus.cc:740] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.081117 8544 consensus_queue.cc:260] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.081231 8725 tablet.cc:1722] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:51.081388 8544 ts_tablet_manager.cc:1434] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:51.081589 8544 tablet_bootstrap.cc:492] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.082605 8544 tablet_bootstrap.cc:654] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.083004 8753 tablet.cc:1722] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:51.083055 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_dff2d248a846455eb7bd3f3648f7ceee" } modify_external_catalogs: true
I20251028 02:59:51.083163 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_dff2d248a846455eb7bd3f3648f7ceee" } modify_external_catalogs: true
I20251028 02:59:51.085258 8545 tablet_bootstrap.cc:492] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.085350 8545 ts_tablet_manager.cc:1403] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 02:59:51.085942 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet c5a698db1042424bad62bce674b2c44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.086165 8545 raft_consensus.cc:359] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.086275 8545 raft_consensus.cc:385] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.086309 8545 raft_consensus.cc:740] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.086427 8545 consensus_queue.cc:260] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.086489 8631 raft_consensus.cc:493] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.086584 8631 raft_consensus.cc:515] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.086715 8545 ts_tablet_manager.cc:1434] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.086925 8545 tablet_bootstrap.cc:492] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.086984 8631 leader_election.cc:290] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.087374 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 0876f8b4ee634d9d92efe9613b3495be from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.087571 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "482d7541b350426f9ff16448a24daeed" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:51.087903 8218 leader_election.cc:343] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.088088 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet cef31f29cc1a4aaa83624aac96f3da04 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.089214 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "482d7541b350426f9ff16448a24daeed" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:51.089567 8220 leader_election.cc:343] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.089643 8220 leader_election.cc:304] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.090106 8556 raft_consensus.cc:2749] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.090571 8545 tablet_bootstrap.cc:654] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.093062 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 70e625899ec546f7add9948e59d289af
I20251028 02:59:51.093390 7963 catalog_manager.cc:6719] ReplaceTablet: tablet 70e625899ec546f7add9948e59d289af deleted and replaced by tablet 6c6c40605a5a48d79837893443f4ca1b
I20251028 02:59:51.093407 8551 tablet_replica.cc:333] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.093526 8551 raft_consensus.cc:2243] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.093575 8552 ts_tablet_manager.cc:1916] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.093803 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 85a0ee11b35545cca8db8754c0b6b7d7
I20251028 02:59:51.093874 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 70e625899ec546f7add9948e59d289af with delete_type TABLET_DATA_DELETED (replaced by tablet 6c6c40605a5a48d79837893443f4ca1b) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.093892 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet d09229ca82eb49f2b4214b238c3ec097 (table default.Table_1f3665f140a54beebcb5f655ce120846 [id=7fe387bad5e740429e327d42f1efde22]). Expected version 2 got 1
I20251028 02:59:51.094062 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 85a0ee11b35545cca8db8754c0b6b7d7 deleted and replaced by tablet fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:51.094065 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 206b71b0f10c42fd81f291a27fffc8a2
I20251028 02:59:51.095090 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 85a0ee11b35545cca8db8754c0b6b7d7 with delete_type TABLET_DATA_DELETED (replaced by tablet fbc0337747a74e20b22aac654c2ab7cc) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.095117 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 85a0ee11b35545cca8db8754c0b6b7d7 with delete_type TABLET_DATA_DELETED (replaced by tablet fbc0337747a74e20b22aac654c2ab7cc) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.095256 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 206b71b0f10c42fd81f291a27fffc8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.095533 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6be0e9d6942e485798736b7f37a9bac6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.095942 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6be0e9d6942e485798736b7f37a9bac6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.095891 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 206b71b0f10c42fd81f291a27fffc8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.096323 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 70e625899ec546f7add9948e59d289af with delete_type TABLET_DATA_DELETED (replaced by tablet 6c6c40605a5a48d79837893443f4ca1b) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.096475 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 85a0ee11b35545cca8db8754c0b6b7d7 with delete_type TABLET_DATA_DELETED (replaced by tablet fbc0337747a74e20b22aac654c2ab7cc) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.097347 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet d09229ca82eb49f2b4214b238c3ec097 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.097831 8552 ts_tablet_manager.cc:1929] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:51.097896 8552 log.cc:1199] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/2386ea3c08fe48c381c7fc2321ecbc86
I20251028 02:59:51.098018 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 206b71b0f10c42fd81f291a27fffc8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.098150 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 70e625899ec546f7add9948e59d289af with delete_type TABLET_DATA_DELETED (replaced by tablet 6c6c40605a5a48d79837893443f4ca1b) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.098172 8552 ts_tablet_manager.cc:1950] T 2386ea3c08fe48c381c7fc2321ecbc86 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.098852 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 2386ea3c08fe48c381c7fc2321ecbc86 (table default.Table_5036a52b716149b2a336ce9c3a019ed1 [id=9d19f4b99b02446c8bbc1263a9b70c4e]) successfully deleted
I20251028 02:59:51.099346 8551 raft_consensus.cc:2272] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.099720 8551 ts_tablet_manager.cc:1916] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.100076 8726 tablet.cc:1722] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:51.101724 8551 ts_tablet_manager.cc:1929] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.101814 8551 log.cc:1199] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c4b5b1d7c738413f80b8aed09b597c60
I20251028 02:59:51.102022 8551 ts_tablet_manager.cc:1950] T c4b5b1d7c738413f80b8aed09b597c60 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.102669 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet c4b5b1d7c738413f80b8aed09b597c60 (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:51.106464 8545 tablet_bootstrap.cc:492] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.106568 8545 ts_tablet_manager.cc:1403] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251028 02:59:51.107062 8545 raft_consensus.cc:359] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.107167 8545 raft_consensus.cc:385] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.107197 8545 raft_consensus.cc:740] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.107336 8545 consensus_queue.cc:260] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.107645 8545 ts_tablet_manager.cc:1434] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.107857 8545 tablet_bootstrap.cc:492] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.108052 8544 tablet_bootstrap.cc:492] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.108152 8544 ts_tablet_manager.cc:1403] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20251028 02:59:51.108650 8544 raft_consensus.cc:359] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.108772 8544 raft_consensus.cc:385] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.108811 8544 raft_consensus.cc:740] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.108942 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 0876f8b4ee634d9d92efe9613b3495be
I20251028 02:59:51.108940 8544 consensus_queue.cc:260] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.109148 8549 raft_consensus.cc:493] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.109215 8544 ts_tablet_manager.cc:1434] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:51.109230 7967 catalog_manager.cc:6719] ReplaceTablet: tablet 0876f8b4ee634d9d92efe9613b3495be deleted and replaced by tablet ab1ec08da9f64109a3fe9400fe4e366c
I20251028 02:59:51.109220 8549 raft_consensus.cc:515] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.109411 8544 tablet_bootstrap.cc:492] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.109555 8549 leader_election.cc:290] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:51.109823 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 0876f8b4ee634d9d92efe9613b3495be with delete_type TABLET_DATA_DELETED (replaced by tablet ab1ec08da9f64109a3fe9400fe4e366c) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.109973 8545 tablet_bootstrap.cc:654] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.110363 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de87fed15e1f4e41b0d962aa858e3128" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.110459 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 0876f8b4ee634d9d92efe9613b3495be with delete_type TABLET_DATA_DELETED (replaced by tablet ab1ec08da9f64109a3fe9400fe4e366c) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.110473 8147 raft_consensus.cc:2468] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.110514 8544 tablet_bootstrap.cc:654] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.110749 8354 leader_election.cc:304] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
W20251028 02:59:51.111550 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ae2927be5f3a440286ee0fe5ce818018 because the tablet was not found. No further retry: Not found: Tablet not found: ae2927be5f3a440286ee0fe5ce818018
I20251028 02:59:51.111769 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de87fed15e1f4e41b0d962aa858e3128" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.111917 8285 raft_consensus.cc:2468] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
W20251028 02:59:51.111963 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 0a3d0d8a0e5e4e34b97230c97b5f27b5 because the tablet was not found. No further retry: Not found: Tablet not found: 0a3d0d8a0e5e4e34b97230c97b5f27b5
I20251028 02:59:51.112142 8551 tablet_replica.cc:333] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.112285 8551 raft_consensus.cc:2243] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.112389 8551 raft_consensus.cc:2272] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.112797 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_828e8df9af6a48168a189935f63b2941"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.112960 8551 ts_tablet_manager.cc:1916] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.113149 8648 raft_consensus.cc:2804] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.113204 8648 raft_consensus.cc:493] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20251028 02:59:51.113206 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_828e8df9af6a48168a189935f63b2941 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.113246 8648 raft_consensus.cc:3060] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.114080 8648 raft_consensus.cc:515] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.114529 8648 leader_election.cc:290] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:51.115159 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de87fed15e1f4e41b0d962aa858e3128" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:51.115272 8147 raft_consensus.cc:3060] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.115957 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_153da548190a42a78cad7ae8553d6fba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.116290 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d09229ca82eb49f2b4214b238c3ec097
I20251028 02:59:51.116318 8147 raft_consensus.cc:2468] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
W20251028 02:59:51.116328 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_153da548190a42a78cad7ae8553d6fba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.116595 7962 catalog_manager.cc:6719] ReplaceTablet: tablet d09229ca82eb49f2b4214b238c3ec097 deleted and replaced by tablet 11a8056e97ff452e93e91622fedb77a6
I20251028 02:59:51.116626 8354 leader_election.cc:304] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.117195 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet d09229ca82eb49f2b4214b238c3ec097 with delete_type TABLET_DATA_DELETED (replaced by tablet 11a8056e97ff452e93e91622fedb77a6) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.117262 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet cef31f29cc1a4aaa83624aac96f3da04
I20251028 02:59:51.117507 7960 catalog_manager.cc:6719] ReplaceTablet: tablet cef31f29cc1a4aaa83624aac96f3da04 deleted and replaced by tablet bd3cebaf966245b7b009b7dd90e2eb71
I20251028 02:59:51.118003 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet cef31f29cc1a4aaa83624aac96f3da04 with delete_type TABLET_DATA_DELETED (replaced by tablet bd3cebaf966245b7b009b7dd90e2eb71) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.118005 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de87fed15e1f4e41b0d962aa858e3128" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.118134 8285 raft_consensus.cc:3060] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.119069 8285 raft_consensus.cc:2468] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.119472 8551 ts_tablet_manager.cc:1929] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:51.119709 8551 log.cc:1199] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2386ea3c08fe48c381c7fc2321ecbc86
I20251028 02:59:51.119732 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 0876f8b4ee634d9d92efe9613b3495be with delete_type TABLET_DATA_DELETED (replaced by tablet ab1ec08da9f64109a3fe9400fe4e366c) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.119623 8648 raft_consensus.cc:2804] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.120040 8648 raft_consensus.cc:697] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:51.120186 8648 consensus_queue.cc:237] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.120747 8546 tablet_bootstrap.cc:492] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.120851 8546 ts_tablet_manager.cc:1403] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.000s
I20251028 02:59:51.121358 8546 raft_consensus.cc:359] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.121476 8546 raft_consensus.cc:385] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.121507 8546 raft_consensus.cc:740] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.121695 8656 raft_consensus.cc:493] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.121800 8656 raft_consensus.cc:515] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.122097 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet d09229ca82eb49f2b4214b238c3ec097 with delete_type TABLET_DATA_DELETED (replaced by tablet 11a8056e97ff452e93e91622fedb77a6) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.122215 8656 leader_election.cc:290] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.122231 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet cef31f29cc1a4aaa83624aac96f3da04 with delete_type TABLET_DATA_DELETED (replaced by tablet bd3cebaf966245b7b009b7dd90e2eb71) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.120040 8551 ts_tablet_manager.cc:1950] T 2386ea3c08fe48c381c7fc2321ecbc86 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.122872 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_966ea54a4f6c408cb6386c3f729e9a9a" } new_table_name: "default.Table_6997dbe5002f4b0594291518fccd3c96" modify_external_catalogs: true
I20251028 02:59:51.122918 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet cef31f29cc1a4aaa83624aac96f3da04 with delete_type TABLET_DATA_DELETED (replaced by tablet bd3cebaf966245b7b009b7dd90e2eb71) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.122944 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "36d7cf962de845cfaca691be18a1bd63" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:51.122987 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2386ea3c08fe48c381c7fc2321ecbc86 (table default.Table_5036a52b716149b2a336ce9c3a019ed1 [id=9d19f4b99b02446c8bbc1263a9b70c4e]) successfully deleted
I20251028 02:59:51.123185 8551 tablet_replica.cc:333] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.123217 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "36d7cf962de845cfaca691be18a1bd63" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.123332 8551 raft_consensus.cc:2243] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.123389 8285 raft_consensus.cc:2393] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 02:59:51.123466 8551 pending_rounds.cc:70] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:51.123492 8551 pending_rounds.cc:77] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597121083994112 op_type: NO_OP noop_request { }
I20251028 02:59:51.123540 8551 raft_consensus.cc:2889] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:51.123589 8551 raft_consensus.cc:2272] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.123579 8546 consensus_queue.cc:260] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.123728 8083 leader_election.cc:304] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:51.124186 8551 ts_tablet_manager.cc:1916] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.124480 8656 raft_consensus.cc:3060] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.125455 8656 raft_consensus.cc:2749] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.125921 8546 ts_tablet_manager.cc:1434] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 02:59:51.126145 8546 tablet_bootstrap.cc:492] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.126457 8551 ts_tablet_manager.cc:1929] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:51.126528 8551 log.cc:1199] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/da5a607a1e7f4ee9b7468b7a12b5a484
I20251028 02:59:51.126794 8551 ts_tablet_manager.cc:1950] T da5a607a1e7f4ee9b7468b7a12b5a484 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.127529 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet da5a607a1e7f4ee9b7468b7a12b5a484 (table default.Table_bf33aa6089da40008abddb5fe2796251 [id=dfeb1ac7ead5405c811d04273658497b]) successfully deleted
I20251028 02:59:51.127606 8551 tablet_replica.cc:333] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.127725 8551 raft_consensus.cc:2243] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
W20251028 02:59:51.127717 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet c5a698db1042424bad62bce674b2c44d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5a698db1042424bad62bce674b2c44d already in progress: creating tablet
I20251028 02:59:51.127715 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet d09229ca82eb49f2b4214b238c3ec097 with delete_type TABLET_DATA_DELETED (replaced by tablet 11a8056e97ff452e93e91622fedb77a6) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.127935 8551 raft_consensus.cc:2272] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.128436 8551 ts_tablet_manager.cc:1916] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.129046 8546 tablet_bootstrap.cc:654] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.131281 8551 ts_tablet_manager.cc:1929] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:51.131558 8551 log.cc:1199] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/85a0ee11b35545cca8db8754c0b6b7d7
I20251028 02:59:51.131923 8551 ts_tablet_manager.cc:1950] T 85a0ee11b35545cca8db8754c0b6b7d7 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.132745 8551 tablet_replica.cc:333] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.132866 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 85a0ee11b35545cca8db8754c0b6b7d7 (table default.Table_1cdf4b1794504307906ac65e49eb2c71 [id=b34ab58e43e943909c9bdd79ad69e639]) successfully deleted
I20251028 02:59:51.132920 8551 raft_consensus.cc:2243] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.133466 8551 raft_consensus.cc:2272] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.134105 8551 ts_tablet_manager.cc:1916] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.134711 7961 catalog_manager.cc:5649] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:51.134948 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 36d7cf962de845cfaca691be18a1bd63 (table default.Table_596fa65222fb43afbb4d4b620ee0a0c1 [id=4a127e52978a4c8ea8a947478ce5996d]). Expected version 1 got 0
I20251028 02:59:51.136420 8122 tablet_service.cc:1505] Processing CreateTablet for tablet a99123e464ae42ca9598a926c537455d (DEFAULT_TABLE table=default.Table_af0da4b12fef412fa59f7ec5fa3fc660 [id=d103d5f71ac24421bb60bf82e44dc5e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.136600 8551 ts_tablet_manager.cc:1929] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:51.136667 8551 log.cc:1199] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/206b71b0f10c42fd81f291a27fffc8a2
I20251028 02:59:51.136729 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a99123e464ae42ca9598a926c537455d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.136957 8551 ts_tablet_manager.cc:1950] T 206b71b0f10c42fd81f291a27fffc8a2 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.137667 8551 tablet_replica.cc:333] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.137785 8551 raft_consensus.cc:2243] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.137879 8551 raft_consensus.cc:2272] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.139024 8127 tablet_service.cc:1505] Processing CreateTablet for tablet d9729143013a4ead9c3144747ecbe58c (DEFAULT_TABLE table=default.Table_d4ddd31975814e4b9c829a132cdc6dad [id=f714f168dbd94c4aaf69df7cb6e4bb3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.139284 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9729143013a4ead9c3144747ecbe58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.139314 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 206b71b0f10c42fd81f291a27fffc8a2 (table default.Table_43b9d6e2a6af4b25ae6e80ecccbe4392 [id=d0c0560068b54a5e98d13b2196c75aa7]) successfully deleted
I20251028 02:59:51.139609 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 662c50d8fa344586bac75614ec20327b (DEFAULT_TABLE table=default.Table_c320b509540344b7895fb967b844a408 [id=fbecd1cf849b47bb880b96a56333fbd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.139832 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 662c50d8fa344586bac75614ec20327b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.142575 8397 tablet_service.cc:1505] Processing CreateTablet for tablet a99123e464ae42ca9598a926c537455d (DEFAULT_TABLE table=default.Table_af0da4b12fef412fa59f7ec5fa3fc660 [id=d103d5f71ac24421bb60bf82e44dc5e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.142861 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a99123e464ae42ca9598a926c537455d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.142937 8631 consensus_queue.cc:1048] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.143044 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 20102243a3514ad1b5dc520b714d2413 (DEFAULT_TABLE table=default.Table_50b58157f999483da285fc8730aba0e9 [id=3f05bbb6510e4149acbb9d0c236ee4f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.143249 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20102243a3514ad1b5dc520b714d2413. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.143529 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 662c50d8fa344586bac75614ec20327b (DEFAULT_TABLE table=default.Table_c320b509540344b7895fb967b844a408 [id=fbecd1cf849b47bb880b96a56333fbd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.143744 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 662c50d8fa344586bac75614ec20327b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.143865 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 9e908d5921d34c9396c1ccdc857c66b7 (DEFAULT_TABLE table=default.Table_6d18f1aded5b45cd85d34e4790ad8d40 [id=4eb2743632814baab2e39bbdce700d6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.144063 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e908d5921d34c9396c1ccdc857c66b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.145841 8401 tablet_service.cc:1505] Processing CreateTablet for tablet d9729143013a4ead9c3144747ecbe58c (DEFAULT_TABLE table=default.Table_d4ddd31975814e4b9c829a132cdc6dad [id=f714f168dbd94c4aaf69df7cb6e4bb3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.146035 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9729143013a4ead9c3144747ecbe58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.148384 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 374936c130c841348e0d76cce070a295 (DEFAULT_TABLE table=default.Table_bb2a8beb93a74d8eb05afe4fb154bbfa [id=90264f1fa3994b8a944e512a95361ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.148612 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 374936c130c841348e0d76cce070a295. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.150046 8260 tablet_service.cc:1505] Processing CreateTablet for tablet a99123e464ae42ca9598a926c537455d (DEFAULT_TABLE table=default.Table_af0da4b12fef412fa59f7ec5fa3fc660 [id=d103d5f71ac24421bb60bf82e44dc5e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.150309 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a99123e464ae42ca9598a926c537455d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.150278 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 662c50d8fa344586bac75614ec20327b (DEFAULT_TABLE table=default.Table_c320b509540344b7895fb967b844a408 [id=fbecd1cf849b47bb880b96a56333fbd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.150452 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 662c50d8fa344586bac75614ec20327b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.150799 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 9e908d5921d34c9396c1ccdc857c66b7 (DEFAULT_TABLE table=default.Table_6d18f1aded5b45cd85d34e4790ad8d40 [id=4eb2743632814baab2e39bbdce700d6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.150993 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e908d5921d34c9396c1ccdc857c66b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.151479 8264 tablet_service.cc:1505] Processing CreateTablet for tablet 025a4d3e71ff443c8bdf649055a17d54 (DEFAULT_TABLE table=default.Table_597d948d86e446488fce9852ca89032d [id=7d237f6c141e4d5e821305500a020dc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.151677 8264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025a4d3e71ff443c8bdf649055a17d54. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:51.151813 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4b4f077b41fc4f8b90db7392984b48a8 consensus state has no leader
I20251028 02:59:51.152642 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 374936c130c841348e0d76cce070a295 (DEFAULT_TABLE table=default.Table_bb2a8beb93a74d8eb05afe4fb154bbfa [id=90264f1fa3994b8a944e512a95361ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.152864 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 374936c130c841348e0d76cce070a295. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.150067 8553 consensus_queue.cc:1048] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20251028 02:59:51.156015 7966 catalog_manager.cc:5931] Not sending DeleteTablet requests; no consensus state for tablet 6c6c40605a5a48d79837893443f4ca1b
I20251028 02:59:51.156558 8550 tablet_replica.cc:333] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.156709 8550 raft_consensus.cc:2243] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.156849 8550 raft_consensus.cc:2272] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.156921 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]). Expected version 1 got 0
I20251028 02:59:51.157042 7965 catalog_manager.cc:5649] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:51.157799 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 20102243a3514ad1b5dc520b714d2413 (DEFAULT_TABLE table=default.Table_50b58157f999483da285fc8730aba0e9 [id=3f05bbb6510e4149acbb9d0c236ee4f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.158053 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20102243a3514ad1b5dc520b714d2413. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.158413 8550 ts_tablet_manager.cc:1916] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.160481 8265 tablet_service.cc:1505] Processing CreateTablet for tablet d9729143013a4ead9c3144747ecbe58c (DEFAULT_TABLE table=default.Table_d4ddd31975814e4b9c829a132cdc6dad [id=f714f168dbd94c4aaf69df7cb6e4bb3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.160810 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9729143013a4ead9c3144747ecbe58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.160981 8544 tablet_bootstrap.cc:492] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.161115 8544 ts_tablet_manager.cc:1403] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.052s user 0.000s sys 0.003s
I20251028 02:59:51.161687 8544 raft_consensus.cc:359] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.161823 8544 raft_consensus.cc:385] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.161865 8544 raft_consensus.cc:740] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.162002 8544 consensus_queue.cc:260] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.162344 8544 ts_tablet_manager.cc:1434] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.162528 8544 tablet_bootstrap.cc:492] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.163267 8552 tablet_replica.cc:333] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.163484 8552 raft_consensus.cc:2243] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.163654 8552 raft_consensus.cc:2272] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.164175 8551 ts_tablet_manager.cc:1916] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.166749 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 025a4d3e71ff443c8bdf649055a17d54 (DEFAULT_TABLE table=default.Table_597d948d86e446488fce9852ca89032d [id=7d237f6c141e4d5e821305500a020dc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.167052 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025a4d3e71ff443c8bdf649055a17d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.167285 8544 tablet_bootstrap.cc:654] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.169270 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 20102243a3514ad1b5dc520b714d2413 (DEFAULT_TABLE table=default.Table_50b58157f999483da285fc8730aba0e9 [id=3f05bbb6510e4149acbb9d0c236ee4f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.169502 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20102243a3514ad1b5dc520b714d2413. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.169690 8552 ts_tablet_manager.cc:1916] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.172010 8552 ts_tablet_manager.cc:1929] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.172098 8552 log.cc:1199] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/da5a607a1e7f4ee9b7468b7a12b5a484
I20251028 02:59:51.172313 8552 ts_tablet_manager.cc:1950] T da5a607a1e7f4ee9b7468b7a12b5a484 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.172613 8546 tablet_bootstrap.cc:492] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.172708 8546 ts_tablet_manager.cc:1403] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20251028 02:59:51.172889 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 374936c130c841348e0d76cce070a295 (DEFAULT_TABLE table=default.Table_bb2a8beb93a74d8eb05afe4fb154bbfa [id=90264f1fa3994b8a944e512a95361ff2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.173012 8552 tablet_replica.cc:333] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.173143 8552 raft_consensus.cc:2243] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.173187 8546 raft_consensus.cc:359] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.173309 8546 raft_consensus.cc:385] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.173244 8552 raft_consensus.cc:2272] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.173345 8546 raft_consensus.cc:740] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.173444 8546 consensus_queue.cc:260] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.173705 8546 ts_tablet_manager.cc:1434] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:51.173980 8546 tablet_bootstrap.cc:492] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.174283 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ec3580a0e3af454f9154d6733e07ca55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.174705 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec3580a0e3af454f9154d6733e07ca55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.174676 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 025a4d3e71ff443c8bdf649055a17d54 (DEFAULT_TABLE table=default.Table_597d948d86e446488fce9852ca89032d [id=7d237f6c141e4d5e821305500a020dc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.174853 8398 tablet_service.cc:1505] Processing CreateTablet for tablet ed7d32bd6b554916af9327befda800e2 (DEFAULT_TABLE table=default.Table_6be0e9d6942e485798736b7f37a9bac6 [id=c103fff3458d4d08affefb7c38d35e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.175127 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7d32bd6b554916af9327befda800e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.175439 8546 tablet_bootstrap.cc:654] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.176504 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 9e908d5921d34c9396c1ccdc857c66b7 (DEFAULT_TABLE table=default.Table_6d18f1aded5b45cd85d34e4790ad8d40 [id=4eb2743632814baab2e39bbdce700d6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.177696 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 2f67b25b85194ce0be43f01adccd6c48 (DEFAULT_TABLE table=default.Table_153da548190a42a78cad7ae8553d6fba [id=9f4f24d46fc048909a3907301bb526de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.177944 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f67b25b85194ce0be43f01adccd6c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.177968 8552 ts_tablet_manager.cc:1916] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.180366 8552 ts_tablet_manager.cc:1929] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.180444 8552 log.cc:1199] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/a3d108be7a734497a9619cd7c097b2e5
I20251028 02:59:51.180668 8552 ts_tablet_manager.cc:1950] T a3d108be7a734497a9619cd7c097b2e5 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.180964 8262 tablet_service.cc:1505] Processing CreateTablet for tablet ed7d32bd6b554916af9327befda800e2 (DEFAULT_TABLE table=default.Table_6be0e9d6942e485798736b7f37a9bac6 [id=c103fff3458d4d08affefb7c38d35e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.181031 8261 tablet_service.cc:1505] Processing CreateTablet for tablet bd3cebaf966245b7b009b7dd90e2eb71 (DEFAULT_TABLE table=default.Table_6532259aa4cc47ee85ab998f23912c02 [id=960dc85e288d4ae6936f98938e123cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.181217 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7d32bd6b554916af9327befda800e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.182010 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 2f67b25b85194ce0be43f01adccd6c48 (DEFAULT_TABLE table=default.Table_153da548190a42a78cad7ae8553d6fba [id=9f4f24d46fc048909a3907301bb526de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.182313 8125 tablet_service.cc:1505] Processing CreateTablet for tablet ed7d32bd6b554916af9327befda800e2 (DEFAULT_TABLE table=default.Table_6be0e9d6942e485798736b7f37a9bac6 [id=c103fff3458d4d08affefb7c38d35e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.182478 8120 tablet_service.cc:1505] Processing CreateTablet for tablet ab1ec08da9f64109a3fe9400fe4e366c (DEFAULT_TABLE table=default.Table_b026ff28518f49aea62b20dd70917a7a [id=7a2a6ab3c9744942a9413c6cb057f527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.182714 8119 tablet_service.cc:1505] Processing CreateTablet for tablet fbc0337747a74e20b22aac654c2ab7cc (DEFAULT_TABLE table=default.Table_1cdf4b1794504307906ac65e49eb2c71 [id=b34ab58e43e943909c9bdd79ad69e639]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.182996 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 6c6c40605a5a48d79837893443f4ca1b (DEFAULT_TABLE table=default.Table_dff2d248a846455eb7bd3f3648f7ceee [id=77dd1eb6df0747bcab57b157ab8a850e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.183281 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 11a8056e97ff452e93e91622fedb77a6 (DEFAULT_TABLE table=default.Table_1f3665f140a54beebcb5f655ce120846 [id=7fe387bad5e740429e327d42f1efde22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.183380 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 2f67b25b85194ce0be43f01adccd6c48 (DEFAULT_TABLE table=default.Table_153da548190a42a78cad7ae8553d6fba [id=9f4f24d46fc048909a3907301bb526de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.183581 8116 tablet_service.cc:1505] Processing CreateTablet for tablet c5971fbd47e64ffda6df61dfc4756dfb (DEFAULT_TABLE table=default.Table_828e8df9af6a48168a189935f63b2941 [id=42c89dfccc714a9eb42e3779130c8af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.182322 8121 tablet_service.cc:1505] Processing CreateTablet for tablet bd3cebaf966245b7b009b7dd90e2eb71 (DEFAULT_TABLE table=default.Table_6532259aa4cc47ee85ab998f23912c02 [id=960dc85e288d4ae6936f98938e123cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.184065 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f67b25b85194ce0be43f01adccd6c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.184254 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet da5a607a1e7f4ee9b7468b7a12b5a484 (table default.Table_bf33aa6089da40008abddb5fe2796251 [id=dfeb1ac7ead5405c811d04273658497b]) successfully deleted
I20251028 02:59:51.185242 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 6c6c40605a5a48d79837893443f4ca1b (DEFAULT_TABLE table=default.Table_dff2d248a846455eb7bd3f3648f7ceee [id=77dd1eb6df0747bcab57b157ab8a850e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.185500 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6c40605a5a48d79837893443f4ca1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.186115 8552 tablet_replica.cc:333] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.186288 8552 raft_consensus.cc:2243] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.186453 8552 raft_consensus.cc:2272] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.187045 8552 ts_tablet_manager.cc:1916] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.187513 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 11a8056e97ff452e93e91622fedb77a6 (DEFAULT_TABLE table=default.Table_1f3665f140a54beebcb5f655ce120846 [id=7fe387bad5e740429e327d42f1efde22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.187762 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11a8056e97ff452e93e91622fedb77a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.189011 8394 tablet_service.cc:1505] Processing CreateTablet for tablet c5971fbd47e64ffda6df61dfc4756dfb (DEFAULT_TABLE table=default.Table_828e8df9af6a48168a189935f63b2941 [id=42c89dfccc714a9eb42e3779130c8af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.189109 8552 ts_tablet_manager.cc:1929] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.189172 8552 log.cc:1199] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/f74e7c0e7d634679bb70cf3515aafe22
I20251028 02:59:51.189246 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5971fbd47e64ffda6df61dfc4756dfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.189399 8552 ts_tablet_manager.cc:1950] T f74e7c0e7d634679bb70cf3515aafe22 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.189882 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3cebaf966245b7b009b7dd90e2eb71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.192283 8551 ts_tablet_manager.cc:1929] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.192358 8551 log.cc:1199] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/70e625899ec546f7add9948e59d289af
I20251028 02:59:51.192371 8552 tablet_replica.cc:333] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.192505 8552 raft_consensus.cc:2243] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.192577 8551 ts_tablet_manager.cc:1950] T 70e625899ec546f7add9948e59d289af P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.192649 8552 pending_rounds.cc:70] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:51.192742 8552 pending_rounds.cc:77] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597121444057088 op_type: NO_OP noop_request { }
I20251028 02:59:51.192811 8552 raft_consensus.cc:2889] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:51.192866 8552 raft_consensus.cc:2272] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.193383 8552 ts_tablet_manager.cc:1916] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.193944 8261 tablet_service.cc:1505] Processing CreateTablet for tablet fbc0337747a74e20b22aac654c2ab7cc (DEFAULT_TABLE table=default.Table_1cdf4b1794504307906ac65e49eb2c71 [id=b34ab58e43e943909c9bdd79ad69e639]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.193975 8262 tablet_service.cc:1505] Processing CreateTablet for tablet ab1ec08da9f64109a3fe9400fe4e366c (DEFAULT_TABLE table=default.Table_b026ff28518f49aea62b20dd70917a7a [id=7a2a6ab3c9744942a9413c6cb057f527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.194201 8550 ts_tablet_manager.cc:1929] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:51.194271 8550 log.cc:1199] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/85a0ee11b35545cca8db8754c0b6b7d7
I20251028 02:59:51.194283 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 6c6c40605a5a48d79837893443f4ca1b (DEFAULT_TABLE table=default.Table_dff2d248a846455eb7bd3f3648f7ceee [id=77dd1eb6df0747bcab57b157ab8a850e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.194514 8550 ts_tablet_manager.cc:1950] T 85a0ee11b35545cca8db8754c0b6b7d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.194653 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 11a8056e97ff452e93e91622fedb77a6 (DEFAULT_TABLE table=default.Table_1f3665f140a54beebcb5f655ce120846 [id=7fe387bad5e740429e327d42f1efde22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.194932 8264 tablet_service.cc:1505] Processing CreateTablet for tablet c5971fbd47e64ffda6df61dfc4756dfb (DEFAULT_TABLE table=default.Table_828e8df9af6a48168a189935f63b2941 [id=42c89dfccc714a9eb42e3779130c8af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.195205 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbc0337747a74e20b22aac654c2ab7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.196118 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]). Expected version 1 got 0
I20251028 02:59:51.196158 8552 ts_tablet_manager.cc:1929] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:51.196213 8552 log.cc:1199] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/70e625899ec546f7add9948e59d289af
I20251028 02:59:51.196242 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 4b4f077b41fc4f8b90db7392984b48a8 (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 1 got 0
I20251028 02:59:51.197154 8552 ts_tablet_manager.cc:1950] T 70e625899ec546f7add9948e59d289af P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.197129 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_809e808b3698494faa7ad0258f9d5774" } modify_external_catalogs: true
I20251028 02:59:51.197247 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_809e808b3698494faa7ad0258f9d5774" } modify_external_catalogs: true
I20251028 02:59:51.197763 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bb2a8beb93a74d8eb05afe4fb154bbfa" } modify_external_catalogs: true
I20251028 02:59:51.197861 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bb2a8beb93a74d8eb05afe4fb154bbfa" } modify_external_catalogs: true
I20251028 02:59:51.199481 8397 tablet_service.cc:1505] Processing CreateTablet for tablet ab1ec08da9f64109a3fe9400fe4e366c (DEFAULT_TABLE table=default.Table_b026ff28518f49aea62b20dd70917a7a [id=7a2a6ab3c9744942a9413c6cb057f527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.199779 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1ec08da9f64109a3fe9400fe4e366c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.199930 8544 tablet_bootstrap.cc:492] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.199987 8545 tablet_bootstrap.cc:492] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.200054 8544 ts_tablet_manager.cc:1403] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.001s
I20251028 02:59:51.200079 8545 ts_tablet_manager.cc:1403] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.092s user 0.002s sys 0.000s
I20251028 02:59:51.200513 8545 raft_consensus.cc:359] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.200625 8545 raft_consensus.cc:385] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.200584 8544 raft_consensus.cc:359] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.200670 8545 raft_consensus.cc:740] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.200708 8544 raft_consensus.cc:385] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.200757 8544 raft_consensus.cc:740] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.200809 8545 consensus_queue.cc:260] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.200902 8544 consensus_queue.cc:260] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.201129 8545 ts_tablet_manager.cc:1434] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.201200 8544 ts_tablet_manager.cc:1434] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:51.201351 8545 tablet_bootstrap.cc:492] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.202349 8765 mvcc.cc:204] Tried to move back new op lower bound from 7215597120288923648 to 7215597118189735936. Current Snapshot: MvccSnapshot[applied={T|T < 7215597120136392704 or (T in {7215597120139251712})}]
I20251028 02:59:51.202428 8545 tablet_bootstrap.cc:654] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.203099 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet a3d108be7a734497a9619cd7c097b2e5 (table default.Table_ddb39e36d68643c2909826b661af4128 [id=8f89769a43034f7d9f783602532e7fef]) successfully deleted
I20251028 02:59:51.203126 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_dddd2e3cad6f4d96bd57539e6195677a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.203420 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet f74e7c0e7d634679bb70cf3515aafe22 (table default.Table_13e18d2c103343648ddade08c263f084 [id=93f5552c487f4ad49d4a1cdf54ea08e7]) successfully deleted
W20251028 02:59:51.203554 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dddd2e3cad6f4d96bd57539e6195677a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.203749 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b026ff28518f49aea62b20dd70917a7a" } new_table_name: "default.Table_97bcd2a2dcfa4b08af27c1db65c4e4e2" modify_external_catalogs: true
I20251028 02:59:51.204201 8398 tablet_service.cc:1505] Processing CreateTablet for tablet fbc0337747a74e20b22aac654c2ab7cc (DEFAULT_TABLE table=default.Table_1cdf4b1794504307906ac65e49eb2c71 [id=b34ab58e43e943909c9bdd79ad69e639]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.204476 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbc0337747a74e20b22aac654c2ab7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.206419 8782 tablet.cc:1722] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.206960 8400 tablet_service.cc:1505] Processing CreateTablet for tablet bd3cebaf966245b7b009b7dd90e2eb71 (DEFAULT_TABLE table=default.Table_6532259aa4cc47ee85ab998f23912c02 [id=960dc85e288d4ae6936f98938e123cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.207198 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3cebaf966245b7b009b7dd90e2eb71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.209889 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11a8056e97ff452e93e91622fedb77a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.211004 8753 tablet.cc:1722] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.212607 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6c40605a5a48d79837893443f4ca1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.213814 8551 tablet_replica.cc:333] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.213941 8551 raft_consensus.cc:2243] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.214078 8551 pending_rounds.cc:70] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:51.214107 8551 pending_rounds.cc:77] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597121022648320 op_type: NO_OP noop_request { }
I20251028 02:59:51.214156 8551 raft_consensus.cc:2889] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:51.214213 8551 raft_consensus.cc:2272] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.214793 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1ec08da9f64109a3fe9400fe4e366c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.217100 8264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5971fbd47e64ffda6df61dfc4756dfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.219081 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 206b71b0f10c42fd81f291a27fffc8a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.220213 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f67b25b85194ce0be43f01adccd6c48. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:51.203568 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet c5a698db1042424bad62bce674b2c44d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5a698db1042424bad62bce674b2c44d already in progress: creating tablet
I20251028 02:59:51.221103 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_60b178c0b65e441d9b23d8b4bc53dedb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.221158 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_eba1e050b6d54c418fbfec1ffcfc8052"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.221478 7955 catalog_manager.cc:5064] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): alter failed for tablet 0876f8b4ee634d9d92efe9613b3495be (table default.Table_b026ff28518f49aea62b20dd70917a7a [id=7a2a6ab3c9744942a9413c6cb057f527]): Illegal state: Tablet not RUNNING: STOPPING
W20251028 02:59:51.221536 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eba1e050b6d54c418fbfec1ffcfc8052 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:51.221542 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60b178c0b65e441d9b23d8b4bc53dedb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.221645 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d012c9d921f542c4ab7a5e8858d351d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.221957 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d012c9d921f542c4ab7a5e8858d351d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.222092 8726 tablet.cc:1722] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.222154 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c2f3577aa7544c82997c62b45e5bb823"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.222435 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2f3577aa7544c82997c62b45e5bb823 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.223394 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 990a32edd54d4d85bcb6108cfb5be4d7
I20251028 02:59:51.223776 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a72c82e0690d484c952d3a1a78883bc5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.224074 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a72c82e0690d484c952d3a1a78883bc5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.225291 8544 tablet_bootstrap.cc:492] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.225468 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e908d5921d34c9396c1ccdc857c66b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.226359 8544 tablet_bootstrap.cc:654] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.226477 8546 tablet_bootstrap.cc:492] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.226548 8546 ts_tablet_manager.cc:1403] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.053s user 0.001s sys 0.001s
I20251028 02:59:51.226971 8546 raft_consensus.cc:359] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.227101 8546 raft_consensus.cc:385] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.227137 8546 raft_consensus.cc:740] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.227274 8546 consensus_queue.cc:260] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.227483 8544 tablet_bootstrap.cc:492] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.227566 8544 ts_tablet_manager.cc:1403] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:51.227627 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7d32bd6b554916af9327befda800e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.227629 8546 ts_tablet_manager.cc:1434] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.227859 8546 tablet_bootstrap.cc:492] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.228492 8544 raft_consensus.cc:359] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.228585 8544 raft_consensus.cc:385] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.228615 8544 raft_consensus.cc:740] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.228750 8544 consensus_queue.cc:260] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.229044 8544 ts_tablet_manager.cc:1434] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.229218 8544 tablet_bootstrap.cc:492] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.229660 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab1ec08da9f64109a3fe9400fe4e366c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.230238 8544 tablet_bootstrap.cc:654] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.230379 8552 tablet_replica.cc:333] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.230492 8546 tablet_bootstrap.cc:654] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.230613 8552 raft_consensus.cc:2243] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.230777 8552 raft_consensus.cc:2272] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.231438 8552 ts_tablet_manager.cc:1916] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.232183 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbc0337747a74e20b22aac654c2ab7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.232851 8551 ts_tablet_manager.cc:1916] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.233888 8546 tablet_bootstrap.cc:492] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.233953 8546 ts_tablet_manager.cc:1403] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20251028 02:59:51.234156 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6c40605a5a48d79837893443f4ca1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.234340 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd3cebaf966245b7b009b7dd90e2eb71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.234460 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 374936c130c841348e0d76cce070a295. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.234555 8546 raft_consensus.cc:359] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.234647 8546 raft_consensus.cc:385] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.234679 8546 raft_consensus.cc:740] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.234853 8546 consensus_queue.cc:260] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.236411 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5971fbd47e64ffda6df61dfc4756dfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.236526 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025a4d3e71ff443c8bdf649055a17d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.236651 8546 ts_tablet_manager.cc:1434] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:51.236774 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 4b4f077b41fc4f8b90db7392984b48a8 (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 1 got 0
I20251028 02:59:51.238219 8546 tablet_bootstrap.cc:492] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.238410 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11a8056e97ff452e93e91622fedb77a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.239351 8546 tablet_bootstrap.cc:654] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.240546 8545 tablet_bootstrap.cc:492] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.240646 8545 ts_tablet_manager.cc:1403] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20251028 02:59:51.241146 8545 raft_consensus.cc:359] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.241258 8545 raft_consensus.cc:385] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.241298 8545 raft_consensus.cc:740] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.241425 8545 consensus_queue.cc:260] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.241724 8545 ts_tablet_manager.cc:1434] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.241950 8545 tablet_bootstrap.cc:492] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.242899 8545 tablet_bootstrap.cc:654] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.242977 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 990a32edd54d4d85bcb6108cfb5be4d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.243639 8550 tablet_replica.cc:333] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.243842 8550 raft_consensus.cc:2243] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.244117 8550 raft_consensus.cc:2272] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.244230 8552 ts_tablet_manager.cc:1929] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:51.244309 8552 log.cc:1199] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/85a0ee11b35545cca8db8754c0b6b7d7
I20251028 02:59:51.244424 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 85a0ee11b35545cca8db8754c0b6b7d7 (table default.Table_1cdf4b1794504307906ac65e49eb2c71 [id=b34ab58e43e943909c9bdd79ad69e639]) successfully deleted
I20251028 02:59:51.244592 8552 ts_tablet_manager.cc:1950] T 85a0ee11b35545cca8db8754c0b6b7d7 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.242893 8551 ts_tablet_manager.cc:1929] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:51.245262 8551 log.cc:1199] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/0876f8b4ee634d9d92efe9613b3495be
I20251028 02:59:51.245509 8551 ts_tablet_manager.cc:1950] T 0876f8b4ee634d9d92efe9613b3495be P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.246179 8551 tablet_replica.cc:333] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.246322 8551 raft_consensus.cc:2243] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.246441 8551 raft_consensus.cc:2272] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.247000 8551 ts_tablet_manager.cc:1916] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.247278 8552 tablet_replica.cc:333] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.247465 8552 raft_consensus.cc:2243] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.248154 8552 raft_consensus.cc:2272] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.248777 8552 ts_tablet_manager.cc:1916] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.250574 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 4b4f077b41fc4f8b90db7392984b48a8 (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 1 got 0
W20251028 02:59:51.251173 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab1ec08da9f64109a3fe9400fe4e366c consensus state has no leader
I20251028 02:59:51.251593 8550 ts_tablet_manager.cc:1916] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.253211 8546 tablet_bootstrap.cc:492] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.253314 8546 ts_tablet_manager.cc:1403] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 02:59:51.253841 8550 ts_tablet_manager.cc:1929] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:51.253931 8550 log.cc:1199] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/206b71b0f10c42fd81f291a27fffc8a2
I20251028 02:59:51.253890 8546 raft_consensus.cc:359] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.254012 8546 raft_consensus.cc:385] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.254069 8546 raft_consensus.cc:740] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.254211 8550 ts_tablet_manager.cc:1950] T 206b71b0f10c42fd81f291a27fffc8a2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.254251 8546 consensus_queue.cc:260] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.254607 8546 ts_tablet_manager.cc:1434] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:51.254832 8546 tablet_bootstrap.cc:492] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.254961 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 206b71b0f10c42fd81f291a27fffc8a2 (table default.Table_43b9d6e2a6af4b25ae6e80ecccbe4392 [id=d0c0560068b54a5e98d13b2196c75aa7]) successfully deleted
I20251028 02:59:51.255950 8546 tablet_bootstrap.cc:654] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.256081 8656 raft_consensus.cc:493] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.256166 8656 raft_consensus.cc:515] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.256497 8656 leader_election.cc:290] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.257187 8545 tablet_bootstrap.cc:492] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.257285 8545 ts_tablet_manager.cc:1403] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
W20251028 02:59:51.257328 8083 leader_election.cc:343] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: 70e625899ec546f7add9948e59d289af
I20251028 02:59:51.257848 8545 raft_consensus.cc:359] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.257959 8545 raft_consensus.cc:385] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.258000 8545 raft_consensus.cc:740] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
W20251028 02:59:51.258064 8083 leader_election.cc:343] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 70e625899ec546f7add9948e59d289af
I20251028 02:59:51.258124 8083 leader_election.cc:304] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:51.258149 8545 consensus_queue.cc:260] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.258330 8656 raft_consensus.cc:2749] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.258469 8545 ts_tablet_manager.cc:1434] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.258690 8545 tablet_bootstrap.cc:492] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.258749 8544 tablet_bootstrap.cc:492] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.258823 8544 ts_tablet_manager.cc:1403] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20251028 02:59:51.259554 8544 raft_consensus.cc:359] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.259667 8544 raft_consensus.cc:385] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.259670 8552 ts_tablet_manager.cc:1929] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:51.259703 8544 raft_consensus.cc:740] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.259734 8552 log.cc:1199] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/206b71b0f10c42fd81f291a27fffc8a2
I20251028 02:59:51.259781 8545 tablet_bootstrap.cc:654] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.259864 8544 consensus_queue.cc:260] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.259996 8552 ts_tablet_manager.cc:1950] T 206b71b0f10c42fd81f291a27fffc8a2 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.260255 8544 ts_tablet_manager.cc:1434] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.260453 8544 tablet_bootstrap.cc:492] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.261475 8545 tablet_bootstrap.cc:492] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.261518 8656 raft_consensus.cc:493] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.261565 8545 ts_tablet_manager.cc:1403] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:51.261586 8656 raft_consensus.cc:515] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.261814 8544 tablet_bootstrap.cc:654] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.261946 8656 leader_election.cc:290] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.262008 8545 raft_consensus.cc:359] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.262077 8551 ts_tablet_manager.cc:1929] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:51.262117 8545 raft_consensus.cc:385] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.262149 8545 raft_consensus.cc:740] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.262274 8552 tablet_replica.cc:333] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.262271 8545 consensus_queue.cc:260] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.262404 8552 raft_consensus.cc:2243] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.262522 8545 ts_tablet_manager.cc:1434] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.262702 8545 tablet_bootstrap.cc:492] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.263266 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0876f8b4ee634d9d92efe9613b3495be" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.262508 8552 raft_consensus.cc:2272] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.263800 8545 tablet_bootstrap.cc:654] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:51.263988 8083 leader_election.cc:343] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 0876f8b4ee634d9d92efe9613b3495be
W20251028 02:59:51.264513 8083 leader_election.cc:343] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.262151 8551 log.cc:1199] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/d09229ca82eb49f2b4214b238c3ec097
I20251028 02:59:51.264604 8083 leader_election.cc:304] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:51.264828 8656 raft_consensus.cc:2749] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.264891 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 374936c130c841348e0d76cce070a295
I20251028 02:59:51.265165 8551 ts_tablet_manager.cc:1950] T d09229ca82eb49f2b4214b238c3ec097 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.265473 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 374936c130c841348e0d76cce070a295 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.265738 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 990a32edd54d4d85bcb6108cfb5be4d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.265969 8545 tablet_bootstrap.cc:492] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.266043 8545 ts_tablet_manager.cc:1403] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:51.266459 8545 raft_consensus.cc:359] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.266549 8545 raft_consensus.cc:385] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.266582 8545 raft_consensus.cc:740] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.266713 8545 consensus_queue.cc:260] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.266855 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet 990a32edd54d4d85bcb6108cfb5be4d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.266820 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1f3665f140a54beebcb5f655ce120846" } new_table_name: "default.Table_fe2e8997c049473bb57e49992eeec9fa" modify_external_catalogs: true
I20251028 02:59:51.266988 8264 tablet_service.cc:1552] Processing DeleteTablet for tablet 374936c130c841348e0d76cce070a295 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.267534 8147 raft_consensus.cc:1275] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:51.267884 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 374936c130c841348e0d76cce070a295 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.267952 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 0876f8b4ee634d9d92efe9613b3495be (table default.Table_97bcd2a2dcfa4b08af27c1db65c4e4e2 [id=7a2a6ab3c9744942a9413c6cb057f527]) successfully deleted
I20251028 02:59:51.267999 8648 consensus_queue.cc:1048] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.268100 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet d09229ca82eb49f2b4214b238c3ec097 (table default.Table_1f3665f140a54beebcb5f655ce120846 [id=7fe387bad5e740429e327d42f1efde22]) successfully deleted
I20251028 02:59:51.268146 8545 ts_tablet_manager.cc:1434] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:51.268237 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 85a0ee11b35545cca8db8754c0b6b7d7 (table default.Table_1cdf4b1794504307906ac65e49eb2c71 [id=b34ab58e43e943909c9bdd79ad69e639]) successfully deleted
I20251028 02:59:51.268318 8545 tablet_bootstrap.cc:492] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.268330 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 206b71b0f10c42fd81f291a27fffc8a2 (table default.Table_43b9d6e2a6af4b25ae6e80ecccbe4392 [id=d0c0560068b54a5e98d13b2196c75aa7]) successfully deleted
I20251028 02:59:51.268500 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_597d948d86e446488fce9852ca89032d" } new_table_name: "default.Table_7218cf040e3a418fa8c1ba0db0d4c1fe" modify_external_catalogs: true
I20251028 02:59:51.269276 8550 tablet_replica.cc:333] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.269322 8545 tablet_bootstrap.cc:654] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.269404 8550 raft_consensus.cc:2243] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.269503 8550 raft_consensus.cc:2272] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.269568 8285 raft_consensus.cc:1275] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:51.270310 8550 ts_tablet_manager.cc:1916] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.270502 8552 ts_tablet_manager.cc:1916] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.272769 8552 ts_tablet_manager.cc:1929] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.272854 8552 log.cc:1199] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/0876f8b4ee634d9d92efe9613b3495be
I20251028 02:59:51.273094 8552 ts_tablet_manager.cc:1950] T 0876f8b4ee634d9d92efe9613b3495be P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.273851 8550 ts_tablet_manager.cc:1929] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.273984 8550 log.cc:1199] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/70e625899ec546f7add9948e59d289af
I20251028 02:59:51.274274 8550 ts_tablet_manager.cc:1950] T 70e625899ec546f7add9948e59d289af P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.275496 7967 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 36d7cf962de845cfaca691be18a1bd63 (table default.Table_596fa65222fb43afbb4d4b620ee0a0c1 [id=4a127e52978a4c8ea8a947478ce5996d]). Expected version 1 got 0
I20251028 02:59:51.275611 7967 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]). Expected version 1 got 0
I20251028 02:59:51.276093 7969 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3] alter complete (version 1)
I20251028 02:59:51.276753 8648 consensus_queue.cc:1048] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.277186 8119 tablet_service.cc:1505] Processing CreateTablet for tablet f11c46ed489e41d3a25cfcd9413443af (DEFAULT_TABLE table=default.Table_ec3580a0e3af454f9154d6733e07ca55 [id=3eb0b70129be40e598874d492735ba51]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.277473 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11c46ed489e41d3a25cfcd9413443af. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:51.279776 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 11a8056e97ff452e93e91622fedb77a6 consensus state has no leader
I20251028 02:59:51.282141 8648 raft_consensus.cc:493] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.282250 8648 raft_consensus.cc:515] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.282649 8648 leader_election.cc:290] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.283201 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cef31f29cc1a4aaa83624aac96f3da04" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.283387 8146 raft_consensus.cc:2468] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.283653 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cef31f29cc1a4aaa83624aac96f3da04" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.283775 8284 raft_consensus.cc:2468] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.283810 8354 leader_election.cc:304] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.284283 8648 raft_consensus.cc:2804] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.284345 8648 raft_consensus.cc:493] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.284385 8648 raft_consensus.cc:3060] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.285086 8656 raft_consensus.cc:493] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.285172 8656 raft_consensus.cc:515] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:51.285313 7953 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet ab1ec08da9f64109a3fe9400fe4e366c (table default.Table_97bcd2a2dcfa4b08af27c1db65c4e4e2 [id=7a2a6ab3c9744942a9413c6cb057f527], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ab1ec08da9f64109a3fe9400fe4e366c consensus state has no leader
I20251028 02:59:51.285435 8648 raft_consensus.cc:515] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.285482 8753 tablet.cc:1722] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.285552 8656 leader_election.cc:290] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.285758 8648 leader_election.cc:290] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.286171 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cef31f29cc1a4aaa83624aac96f3da04" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:51.286270 8146 raft_consensus.cc:3060] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.287235 8146 raft_consensus.cc:2468] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.288008 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e7dbafe267a4c62b0418d1b26585aad" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.288007 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e7dbafe267a4c62b0418d1b26585aad" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:51.288166 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cef31f29cc1a4aaa83624aac96f3da04" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.288270 8283 raft_consensus.cc:3060] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.288453 8782 tablet.cc:1722] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251028 02:59:51.288507 8083 leader_election.cc:343] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:51.288655 8083 leader_election.cc:343] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.288686 8354 leader_election.cc:304] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.288708 8083 leader_election.cc:304] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:51.288887 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b0abfe5364749bd86b9916480828ffd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:50 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.288911 8656 raft_consensus.cc:2749] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.289070 8616 raft_consensus.cc:2804] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.289149 8616 raft_consensus.cc:697] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:51.289291 8616 consensus_queue.cc:237] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.289361 8283 raft_consensus.cc:2468] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.298959 8551 tablet_replica.cc:333] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.299235 8551 raft_consensus.cc:2243] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.299489 8551 pending_rounds.cc:70] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:51.299592 8551 pending_rounds.cc:77] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597122722549760 op_type: NO_OP noop_request { }
I20251028 02:59:51.299728 8551 raft_consensus.cc:2889] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:51.299888 8551 raft_consensus.cc:2272] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.302217 8544 tablet_bootstrap.cc:492] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.302331 8544 ts_tablet_manager.cc:1403] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.000s
I20251028 02:59:51.302865 8544 raft_consensus.cc:359] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.302984 8544 raft_consensus.cc:385] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.303023 8544 raft_consensus.cc:740] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.303458 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 3e7dbafe267a4c62b0418d1b26585aad (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:51.303596 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 36d7cf962de845cfaca691be18a1bd63 (table default.Table_596fa65222fb43afbb4d4b620ee0a0c1 [id=4a127e52978a4c8ea8a947478ce5996d]). Expected version 1 got 0
W20251028 02:59:51.303855 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 025a4d3e71ff443c8bdf649055a17d54 consensus state has no leader
W20251028 02:59:51.304236 7961 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet 3e7dbafe267a4c62b0418d1b26585aad (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 3e7dbafe267a4c62b0418d1b26585aad consensus state has no leader
I20251028 02:59:51.304585 8544 consensus_queue.cc:260] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.305016 8544 ts_tablet_manager.cc:1434] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:51.306953 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ca53be73e3384c6ebb65c41f31a6a4b1" } modify_external_catalogs: true
I20251028 02:59:51.307092 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ca53be73e3384c6ebb65c41f31a6a4b1" } modify_external_catalogs: true
I20251028 02:59:51.307539 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 4132a2b416cc434fbaebfd7df7e5d878 (DEFAULT_TABLE table=default.Table_60b178c0b65e441d9b23d8b4bc53dedb [id=4d9d3f97f0d541f7a90885b9ab852d83]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.307844 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4132a2b416cc434fbaebfd7df7e5d878. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.307876 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet ab1ec08da9f64109a3fe9400fe4e366c from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.308171 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 71909cad34b044b093a94397c4b9f29c (DEFAULT_TABLE table=default.Table_d012c9d921f542c4ab7a5e8858d351d4 [id=739f0673774d472bbb62f176b70998bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.308454 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71909cad34b044b093a94397c4b9f29c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.308743 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 63ee5aa5a6e841fdb9244563d75394ac (DEFAULT_TABLE table=default.Table_c2f3577aa7544c82997c62b45e5bb823 [id=79b638b293df431cbd580e62d24e8238]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.308946 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63ee5aa5a6e841fdb9244563d75394ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.310390 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 71909cad34b044b093a94397c4b9f29c (DEFAULT_TABLE table=default.Table_d012c9d921f542c4ab7a5e8858d351d4 [id=739f0673774d472bbb62f176b70998bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.310626 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71909cad34b044b093a94397c4b9f29c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.310750 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 66fb0173e40e40999f90f015ddb294b7 (DEFAULT_TABLE table=default.Table_eba1e050b6d54c418fbfec1ffcfc8052 [id=d2babb92c2514978b979b30ab6c511f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.310851 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet 662c50d8fa344586bac75614ec20327b from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.310976 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66fb0173e40e40999f90f015ddb294b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.312915 8397 tablet_service.cc:1505] Processing CreateTablet for tablet b1b505f76ef84af6bc2b78a962bf6cf9 (DEFAULT_TABLE table=default.Table_a72c82e0690d484c952d3a1a78883bc5 [id=32b09554787b45c0811966bfb3f70031]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.313076 8116 tablet_service.cc:1505] Processing CreateTablet for tablet b1b505f76ef84af6bc2b78a962bf6cf9 (DEFAULT_TABLE table=default.Table_a72c82e0690d484c952d3a1a78883bc5 [id=32b09554787b45c0811966bfb3f70031]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.313156 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b505f76ef84af6bc2b78a962bf6cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.313287 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b505f76ef84af6bc2b78a962bf6cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.315760 8264 tablet_service.cc:1505] Processing CreateTablet for tablet f11c46ed489e41d3a25cfcd9413443af (DEFAULT_TABLE table=default.Table_ec3580a0e3af454f9154d6733e07ca55 [id=3eb0b70129be40e598874d492735ba51]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.316067 8264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11c46ed489e41d3a25cfcd9413443af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.316831 8546 tablet_bootstrap.cc:492] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.316817 8726 tablet.cc:1722] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.316916 8546 ts_tablet_manager.cc:1403] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.062s user 0.000s sys 0.002s
I20251028 02:59:51.317487 8546 raft_consensus.cc:359] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.317584 8546 raft_consensus.cc:385] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.317611 8546 raft_consensus.cc:740] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.317729 8546 consensus_queue.cc:260] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.318044 8546 ts_tablet_manager.cc:1434] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.318226 8546 tablet_bootstrap.cc:492] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.318502 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 4132a2b416cc434fbaebfd7df7e5d878 (DEFAULT_TABLE table=default.Table_60b178c0b65e441d9b23d8b4bc53dedb [id=4d9d3f97f0d541f7a90885b9ab852d83]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.318732 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4132a2b416cc434fbaebfd7df7e5d878. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.318852 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b44dd3c16841435a8dd467a1da28db7a" } modify_external_catalogs: true
I20251028 02:59:51.318986 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b44dd3c16841435a8dd467a1da28db7a" } modify_external_catalogs: true
I20251028 02:59:51.319464 8546 tablet_bootstrap.cc:654] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.320858 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 4132a2b416cc434fbaebfd7df7e5d878 (DEFAULT_TABLE table=default.Table_60b178c0b65e441d9b23d8b4bc53dedb [id=4d9d3f97f0d541f7a90885b9ab852d83]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.320869 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 71909cad34b044b093a94397c4b9f29c (DEFAULT_TABLE table=default.Table_d012c9d921f542c4ab7a5e8858d351d4 [id=739f0673774d472bbb62f176b70998bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.321084 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4132a2b416cc434fbaebfd7df7e5d878. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.321091 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71909cad34b044b093a94397c4b9f29c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.323170 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 66fb0173e40e40999f90f015ddb294b7 (DEFAULT_TABLE table=default.Table_eba1e050b6d54c418fbfec1ffcfc8052 [id=d2babb92c2514978b979b30ab6c511f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.323484 8259 tablet_service.cc:1505] Processing CreateTablet for tablet b1b505f76ef84af6bc2b78a962bf6cf9 (DEFAULT_TABLE table=default.Table_a72c82e0690d484c952d3a1a78883bc5 [id=32b09554787b45c0811966bfb3f70031]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.323724 8263 tablet_service.cc:1505] Processing CreateTablet for tablet c24384fecd7b4c0a9b2e3da1e0369866 (DEFAULT_TABLE table=default.Table_dddd2e3cad6f4d96bd57539e6195677a [id=b82f549c8f03449eaa1762408bc67772]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.324033 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 63ee5aa5a6e841fdb9244563d75394ac (DEFAULT_TABLE table=default.Table_c2f3577aa7544c82997c62b45e5bb823 [id=79b638b293df431cbd580e62d24e8238]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.326086 8123 tablet_service.cc:1505] Processing CreateTablet for tablet c24384fecd7b4c0a9b2e3da1e0369866 (DEFAULT_TABLE table=default.Table_dddd2e3cad6f4d96bd57539e6195677a [id=b82f549c8f03449eaa1762408bc67772]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.326334 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24384fecd7b4c0a9b2e3da1e0369866. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.315289 8395 tablet_service.cc:1505] Processing CreateTablet for tablet c24384fecd7b4c0a9b2e3da1e0369866 (DEFAULT_TABLE table=default.Table_dddd2e3cad6f4d96bd57539e6195677a [id=b82f549c8f03449eaa1762408bc67772]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.328635 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24384fecd7b4c0a9b2e3da1e0369866. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.315498 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 63ee5aa5a6e841fdb9244563d75394ac (DEFAULT_TABLE table=default.Table_c2f3577aa7544c82997c62b45e5bb823 [id=79b638b293df431cbd580e62d24e8238]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.328830 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63ee5aa5a6e841fdb9244563d75394ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.329627 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66fb0173e40e40999f90f015ddb294b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.307243 8401 tablet_service.cc:1505] Processing CreateTablet for tablet f11c46ed489e41d3a25cfcd9413443af (DEFAULT_TABLE table=default.Table_ec3580a0e3af454f9154d6733e07ca55 [id=3eb0b70129be40e598874d492735ba51]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.328131 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 66fb0173e40e40999f90f015ddb294b7 (DEFAULT_TABLE table=default.Table_eba1e050b6d54c418fbfec1ffcfc8052 [id=d2babb92c2514978b979b30ab6c511f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.333326 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 662c50d8fa344586bac75614ec20327b
I20251028 02:59:51.333371 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11c46ed489e41d3a25cfcd9413443af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.333632 7963 catalog_manager.cc:6719] ReplaceTablet: tablet 662c50d8fa344586bac75614ec20327b deleted and replaced by tablet 3e5f0326a03040de81a506caa80dcbd9
I20251028 02:59:51.334194 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66fb0173e40e40999f90f015ddb294b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.334657 8550 tablet_replica.cc:333] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.334775 8550 raft_consensus.cc:2243] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.334916 8550 raft_consensus.cc:2272] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.335613 8545 tablet_bootstrap.cc:492] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.335593 8550 ts_tablet_manager.cc:1916] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.335700 8545 ts_tablet_manager.cc:1403] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.067s user 0.002s sys 0.000s
I20251028 02:59:51.336212 8545 raft_consensus.cc:359] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.336330 8545 raft_consensus.cc:385] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.336364 8545 raft_consensus.cc:740] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.337054 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c24384fecd7b4c0a9b2e3da1e0369866. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.337046 8545 consensus_queue.cc:260] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.337452 8545 ts_tablet_manager.cc:1434] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 02:59:51.337626 8545 tablet_bootstrap.cc:492] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.339038 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_15bfc391c65b4379b26b45ef36d49a30" } new_table_name: "default.Table_35fd4c0e400244f28d1a855779611c46" modify_external_catalogs: true
I20251028 02:59:51.339175 8545 tablet_bootstrap.cc:654] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.341080 7969 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_596fa65222fb43afbb4d4b620ee0a0c1 [id=4a127e52978a4c8ea8a947478ce5996d] alter complete (version 1)
I20251028 02:59:51.341331 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet c5a698db1042424bad62bce674b2c44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.341894 8550 ts_tablet_manager.cc:1929] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.341967 8550 log.cc:1199] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/0876f8b4ee634d9d92efe9613b3495be
I20251028 02:59:51.342212 8550 ts_tablet_manager.cc:1950] T 0876f8b4ee634d9d92efe9613b3495be P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.342331 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63ee5aa5a6e841fdb9244563d75394ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.343065 8544 tablet_bootstrap.cc:492] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.343452 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b505f76ef84af6bc2b78a962bf6cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.344445 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 662c50d8fa344586bac75614ec20327b with delete_type TABLET_DATA_DELETED (replaced by tablet 3e5f0326a03040de81a506caa80dcbd9) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.344601 8544 tablet_bootstrap.cc:654] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.344937 8550 tablet_replica.cc:333] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.345039 8550 raft_consensus.cc:2243] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.345134 8550 raft_consensus.cc:2272] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.345679 8544 tablet_bootstrap.cc:492] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.345763 8544 ts_tablet_manager.cc:1403] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:51.345840 8550 ts_tablet_manager.cc:1916] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.346241 8544 raft_consensus.cc:359] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.346331 8544 raft_consensus.cc:385] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.346309 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 0876f8b4ee634d9d92efe9613b3495be (table default.Table_97bcd2a2dcfa4b08af27c1db65c4e4e2 [id=7a2a6ab3c9744942a9413c6cb057f527]) successfully deleted
I20251028 02:59:51.346360 8544 raft_consensus.cc:740] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.346493 8544 consensus_queue.cc:260] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.346839 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4c1aacffea564931bb49e6bc29c6b0b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.347262 8552 tablet_replica.cc:333] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 02:59:51.347277 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c1aacffea564931bb49e6bc29c6b0b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.347393 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 662c50d8fa344586bac75614ec20327b with delete_type TABLET_DATA_DELETED (replaced by tablet 3e5f0326a03040de81a506caa80dcbd9) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.347489 8552 raft_consensus.cc:2243] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.347594 8552 raft_consensus.cc:2272] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.347779 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 0876f8b4ee634d9d92efe9613b3495be (table default.Table_97bcd2a2dcfa4b08af27c1db65c4e4e2 [id=7a2a6ab3c9744942a9413c6cb057f527]) successfully deleted
I20251028 02:59:51.347867 8544 ts_tablet_manager.cc:1434] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:51.348120 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet ed7d32bd6b554916af9327befda800e2 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.348138 8551 ts_tablet_manager.cc:1916] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.348045 8544 tablet_bootstrap.cc:492] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.348814 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_392307bd90eb4fef97bb343213d39e89"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.349015 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet de87fed15e1f4e41b0d962aa858e3128
W20251028 02:59:51.349182 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_392307bd90eb4fef97bb343213d39e89 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.349893 8544 tablet_bootstrap.cc:654] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.349879 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 4b4f077b41fc4f8b90db7392984b48a8 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.350088 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_378281b5d5ed40cda0b93ef4a3094345"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.350411 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_378281b5d5ed40cda0b93ef4a3094345 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.350561 8551 ts_tablet_manager.cc:1929] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:51.350632 8551 log.cc:1199] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/cef31f29cc1a4aaa83624aac96f3da04
I20251028 02:59:51.350890 8551 ts_tablet_manager.cc:1950] T cef31f29cc1a4aaa83624aac96f3da04 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.352551 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet cef31f29cc1a4aaa83624aac96f3da04 (table default.Table_6532259aa4cc47ee85ab998f23912c02 [id=960dc85e288d4ae6936f98938e123cf9]) successfully deleted
I20251028 02:59:51.352612 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet de87fed15e1f4e41b0d962aa858e3128 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:51.352708 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 206b71b0f10c42fd81f291a27fffc8a2 because the tablet was not found. No further retry: Not found: Tablet not found: 206b71b0f10c42fd81f291a27fffc8a2
I20251028 02:59:51.352895 8551 tablet_replica.cc:333] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.353046 8551 raft_consensus.cc:2243] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.353057 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet de87fed15e1f4e41b0d962aa858e3128 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.353157 8551 raft_consensus.cc:2272] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.353643 8551 ts_tablet_manager.cc:1916] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.354655 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ab1ec08da9f64109a3fe9400fe4e366c
I20251028 02:59:51.354955 7964 catalog_manager.cc:6719] ReplaceTablet: tablet ab1ec08da9f64109a3fe9400fe4e366c deleted and replaced by tablet 0e20b1f36d3f4f70a93ad8a0b29d6687
I20251028 02:59:51.354985 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 662c50d8fa344586bac75614ec20327b with delete_type TABLET_DATA_DELETED (replaced by tablet 3e5f0326a03040de81a506caa80dcbd9) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.355185 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet de87fed15e1f4e41b0d962aa858e3128 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.355471 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 773b2414ba1c46d7b6982768967d307d
I20251028 02:59:51.355688 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet ab1ec08da9f64109a3fe9400fe4e366c with delete_type TABLET_DATA_DELETED (replaced by tablet 0e20b1f36d3f4f70a93ad8a0b29d6687) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.355880 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet ab1ec08da9f64109a3fe9400fe4e366c with delete_type TABLET_DATA_DELETED (replaced by tablet 0e20b1f36d3f4f70a93ad8a0b29d6687) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.356185 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet ab1ec08da9f64109a3fe9400fe4e366c with delete_type TABLET_DATA_DELETED (replaced by tablet 0e20b1f36d3f4f70a93ad8a0b29d6687) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.356510 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 773b2414ba1c46d7b6982768967d307d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:51.357359 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8643565320504fe6b6154e936e55fca9 consensus state has no leader
I20251028 02:59:51.357527 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet de87fed15e1f4e41b0d962aa858e3128 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.360312 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ed7d32bd6b554916af9327befda800e2
I20251028 02:59:51.360602 7967 catalog_manager.cc:6719] ReplaceTablet: tablet ed7d32bd6b554916af9327befda800e2 deleted and replaced by tablet 1c81698dcbe14d5ba61471e7249705db
I20251028 02:59:51.361063 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 773b2414ba1c46d7b6982768967d307d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.361292 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet ed7d32bd6b554916af9327befda800e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 1c81698dcbe14d5ba61471e7249705db) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.362128 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet c5a698db1042424bad62bce674b2c44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.362309 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 773b2414ba1c46d7b6982768967d307d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.363103 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet ed7d32bd6b554916af9327befda800e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 1c81698dcbe14d5ba61471e7249705db) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.363173 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet ed7d32bd6b554916af9327befda800e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 1c81698dcbe14d5ba61471e7249705db) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.364394 8551 ts_tablet_manager.cc:1929] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.364535 8551 log.cc:1199] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/990a32edd54d4d85bcb6108cfb5be4d7
I20251028 02:59:51.364835 8551 ts_tablet_manager.cc:1950] T 990a32edd54d4d85bcb6108cfb5be4d7 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.365213 8550 ts_tablet_manager.cc:1929] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:51.365301 8550 log.cc:1199] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/d09229ca82eb49f2b4214b238c3ec097
I20251028 02:59:51.365576 8550 ts_tablet_manager.cc:1950] T d09229ca82eb49f2b4214b238c3ec097 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.365808 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 990a32edd54d4d85bcb6108cfb5be4d7 (table default.Table_809e808b3698494faa7ad0258f9d5774 [id=f1e543817e8b48c8bf0e7f5ddb57330c]) successfully deleted
I20251028 02:59:51.366238 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1cdf4b1794504307906ac65e49eb2c71" } modify_external_catalogs: true
I20251028 02:59:51.366365 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1cdf4b1794504307906ac65e49eb2c71" } modify_external_catalogs: true
W20251028 02:59:51.366685 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 374936c130c841348e0d76cce070a295 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 374936c130c841348e0d76cce070a295 already in progress: creating tablet
I20251028 02:59:51.366963 8550 tablet_replica.cc:333] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.367102 8550 raft_consensus.cc:2243] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.367215 8550 raft_consensus.cc:2272] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:51.367439 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 662c50d8fa344586bac75614ec20327b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 662c50d8fa344586bac75614ec20327b already in progress: creating tablet
I20251028 02:59:51.366727 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_392307bd90eb4fef97bb343213d39e89" } new_table_name: "default.Table_458c88f7cba54a43b1553e883083a60b" modify_external_catalogs: true
I20251028 02:59:51.367942 8551 tablet_replica.cc:333] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.367990 8545 tablet_bootstrap.cc:492] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.368074 8545 ts_tablet_manager.cc:1403] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.001s
I20251028 02:59:51.368099 8550 ts_tablet_manager.cc:1916] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.368518 8546 tablet_bootstrap.cc:492] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.368619 8546 ts_tablet_manager.cc:1403] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.050s user 0.000s sys 0.002s
I20251028 02:59:51.368626 8545 raft_consensus.cc:359] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.368732 8545 raft_consensus.cc:385] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.368765 8545 raft_consensus.cc:740] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.368928 8545 consensus_queue.cc:260] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.369217 8545 ts_tablet_manager.cc:1434] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.369174 8546 raft_consensus.cc:359] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.369285 8546 raft_consensus.cc:385] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.369318 8546 raft_consensus.cc:740] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.369417 8545 tablet_bootstrap.cc:492] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.369453 8546 consensus_queue.cc:260] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.369724 8546 ts_tablet_manager.cc:1434] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.369964 8546 tablet_bootstrap.cc:492] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.370512 8545 tablet_bootstrap.cc:654] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.370500 8552 ts_tablet_manager.cc:1916] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.370918 8546 tablet_bootstrap.cc:654] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.371765 8544 tablet_bootstrap.cc:492] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.371876 8544 ts_tablet_manager.cc:1403] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20251028 02:59:51.372400 8544 raft_consensus.cc:359] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.372509 8544 raft_consensus.cc:385] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.372542 8544 raft_consensus.cc:740] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.372702 8544 consensus_queue.cc:260] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.372849 8552 ts_tablet_manager.cc:1929] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.373616 8550 ts_tablet_manager.cc:1929] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.373806 8550 log.cc:1199] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/cef31f29cc1a4aaa83624aac96f3da04
I20251028 02:59:51.373853 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet d09229ca82eb49f2b4214b238c3ec097 (table default.Table_fe2e8997c049473bb57e49992eeec9fa [id=7fe387bad5e740429e327d42f1efde22]) successfully deleted
I20251028 02:59:51.374114 8544 ts_tablet_manager.cc:1434] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:51.374156 8550 ts_tablet_manager.cc:1950] T cef31f29cc1a4aaa83624aac96f3da04 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.374315 8544 tablet_bootstrap.cc:492] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.375113 8616 raft_consensus.cc:493] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.375224 8616 raft_consensus.cc:515] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.375267 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet cef31f29cc1a4aaa83624aac96f3da04 (table default.Table_6532259aa4cc47ee85ab998f23912c02 [id=960dc85e288d4ae6936f98938e123cf9]) successfully deleted
I20251028 02:59:51.375648 8550 tablet_replica.cc:333] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.375698 8616 leader_election.cc:290] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.375892 8550 raft_consensus.cc:2243] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.376067 8550 raft_consensus.cc:2272] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.376215 8544 tablet_bootstrap.cc:654] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.376308 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3209160f7b0e4f1a8c8121725454d088" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:51.376753 8354 leader_election.cc:343] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.377399 8550 ts_tablet_manager.cc:1916] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.376890 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3209160f7b0e4f1a8c8121725454d088" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.378505 8283 raft_consensus.cc:2468] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.368081 8551 raft_consensus.cc:2243] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.378695 8552 log.cc:1199] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/cef31f29cc1a4aaa83624aac96f3da04
I20251028 02:59:51.378846 8356 leader_election.cc:304] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.378952 8552 ts_tablet_manager.cc:1950] T cef31f29cc1a4aaa83624aac96f3da04 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.379146 8616 raft_consensus.cc:2804] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.379211 8616 raft_consensus.cc:493] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.379251 8616 raft_consensus.cc:3060] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.380303 8552 tablet_replica.cc:333] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.380345 8616 raft_consensus.cc:515] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.380445 8552 raft_consensus.cc:2243] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.380621 8552 raft_consensus.cc:2272] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.380740 8616 leader_election.cc:290] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.381027 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3209160f7b0e4f1a8c8121725454d088" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:51.381100 8551 pending_rounds.cc:70] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:51.381142 8551 pending_rounds.cc:77] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597122029965312 op_type: NO_OP noop_request { }
I20251028 02:59:51.381192 8551 raft_consensus.cc:2889] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:51.381188 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3209160f7b0e4f1a8c8121725454d088" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.381246 8551 raft_consensus.cc:2272] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.381286 8283 raft_consensus.cc:3060] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:51.381407 8354 leader_election.cc:343] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.381677 8544 tablet_bootstrap.cc:492] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.381759 8544 ts_tablet_manager.cc:1403] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:51.382225 8283 raft_consensus.cc:2468] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.382282 8544 raft_consensus.cc:359] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.382378 8544 raft_consensus.cc:385] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.382409 8544 raft_consensus.cc:740] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.382555 8544 consensus_queue.cc:260] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.382643 8356 leader_election.cc:304] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.382848 8544 ts_tablet_manager.cc:1434] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.382865 8616 raft_consensus.cc:2804] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.382932 8616 raft_consensus.cc:697] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:51.383046 8616 consensus_queue.cc:237] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.385903 8551 ts_tablet_manager.cc:1916] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.387187 8544 tablet_bootstrap.cc:492] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.387658 8552 ts_tablet_manager.cc:1916] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.388291 8544 tablet_bootstrap.cc:654] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.388679 8551 ts_tablet_manager.cc:1929] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:51.388818 8551 log.cc:1199] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/de87fed15e1f4e41b0d962aa858e3128
I20251028 02:59:51.389102 8551 ts_tablet_manager.cc:1950] T de87fed15e1f4e41b0d962aa858e3128 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.389586 8545 tablet_bootstrap.cc:492] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
W20251028 02:59:51.389628 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f4bebc4575e4462cbc8a02fca4fb228b has no consensus state
I20251028 02:59:51.389719 8545 ts_tablet_manager.cc:1403] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251028 02:59:51.389811 8550 ts_tablet_manager.cc:1929] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.389878 8550 log.cc:1199] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/990a32edd54d4d85bcb6108cfb5be4d7
I20251028 02:59:51.390077 8550 ts_tablet_manager.cc:1950] T 990a32edd54d4d85bcb6108cfb5be4d7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.390305 8545 raft_consensus.cc:359] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.390404 8545 raft_consensus.cc:385] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.390379 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 3e5f0326a03040de81a506caa80dcbd9 (DEFAULT_TABLE table=default.Table_c320b509540344b7895fb967b844a408 [id=fbecd1cf849b47bb880b96a56333fbd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.390435 8545 raft_consensus.cc:740] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.390594 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e5f0326a03040de81a506caa80dcbd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.390563 8545 consensus_queue.cc:260] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.390765 8550 tablet_replica.cc:333] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.390848 8545 ts_tablet_manager.cc:1434] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:51.390887 8550 raft_consensus.cc:2243] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.390980 8550 raft_consensus.cc:2272] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.391040 8545 tablet_bootstrap.cc:492] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.392089 8545 tablet_bootstrap.cc:654] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.392578 8552 ts_tablet_manager.cc:1929] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:51.392652 8552 log.cc:1199] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/d09229ca82eb49f2b4214b238c3ec097
I20251028 02:59:51.392915 8552 ts_tablet_manager.cc:1950] T d09229ca82eb49f2b4214b238c3ec097 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.393909 8656 raft_consensus.cc:493] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.393996 8656 raft_consensus.cc:515] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.394377 8656 leader_election.cc:290] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.394500 8550 ts_tablet_manager.cc:1916] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.395576 8616 raft_consensus.cc:493] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.395664 8616 raft_consensus.cc:515] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.396001 8616 leader_election.cc:290] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.396448 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8643565320504fe6b6154e936e55fca9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:51.396561 8421 raft_consensus.cc:2468] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:51.396692 8551 tablet_replica.cc:333] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.396838 8551 raft_consensus.cc:2243] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.396952 8551 raft_consensus.cc:2272] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.396988 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 990a32edd54d4d85bcb6108cfb5be4d7 (table default.Table_809e808b3698494faa7ad0258f9d5774 [id=f1e543817e8b48c8bf0e7f5ddb57330c]) successfully deleted
I20251028 02:59:51.397079 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5a698db1042424bad62bce674b2c44d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:51.397156 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 374936c130c841348e0d76cce070a295 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 374936c130c841348e0d76cce070a295 already in progress: creating tablet
I20251028 02:59:51.397524 8551 ts_tablet_manager.cc:1916] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:51.397544 8354 leader_election.cc:343] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.397686 8083 leader_election.cc:304] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.393358 8546 tablet_bootstrap.cc:492] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.398411 8546 ts_tablet_manager.cc:1403] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20251028 02:59:51.398586 8656 raft_consensus.cc:2804] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.398646 8656 raft_consensus.cc:493] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.398690 8656 raft_consensus.cc:3060] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.398985 8546 raft_consensus.cc:359] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.399099 8546 raft_consensus.cc:385] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.399132 8546 raft_consensus.cc:740] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.399255 8546 consensus_queue.cc:260] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.399603 8546 ts_tablet_manager.cc:1434] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.394768 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8643565320504fe6b6154e936e55fca9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.399708 8283 raft_consensus.cc:2468] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:51.399725 8656 raft_consensus.cc:515] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.396486 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c5a698db1042424bad62bce674b2c44d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.399891 8285 raft_consensus.cc:2468] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.400158 8656 leader_election.cc:290] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.400166 8356 leader_election.cc:304] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:51.400280 8356 raft_consensus.cc:2686] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251028 02:59:51.400566 8551 ts_tablet_manager.cc:1929] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.400606 8546 tablet_bootstrap.cc:492] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.401247 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8643565320504fe6b6154e936e55fca9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.401350 8285 raft_consensus.cc:3060] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.401655 8546 tablet_bootstrap.cc:654] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.402281 8285 raft_consensus.cc:2468] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:51.402942 8083 leader_election.cc:304] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.404470 8550 ts_tablet_manager.cc:1929] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.404680 8552 tablet_replica.cc:333] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.404870 8552 raft_consensus.cc:2243] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.405036 8552 raft_consensus.cc:2272] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.400985 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8643565320504fe6b6154e936e55fca9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:51.405488 8421 raft_consensus.cc:3060] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.405678 8552 ts_tablet_manager.cc:1916] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.406545 8421 raft_consensus.cc:2468] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:51.406778 8556 raft_consensus.cc:493] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.406873 8556 raft_consensus.cc:515] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.407371 8550 log.cc:1199] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9b0abfe5364749bd86b9916480828ffd
I20251028 02:59:51.407600 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "773b2414ba1c46d7b6982768967d307d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.407262 8556 leader_election.cc:290] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.407732 8147 raft_consensus.cc:2468] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:51.407943 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "773b2414ba1c46d7b6982768967d307d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:51.408053 8218 leader_election.cc:304] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.408082 8421 raft_consensus.cc:2468] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:51.408212 8556 raft_consensus.cc:2804] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.408263 8556 raft_consensus.cc:493] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.408301 8556 raft_consensus.cc:3060] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.405161 8545 tablet_bootstrap.cc:492] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.408401 8545 ts_tablet_manager.cc:1403] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 02:59:51.408586 8656 raft_consensus.cc:2804] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.408679 8656 raft_consensus.cc:697] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:51.408808 8656 consensus_queue.cc:237] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.409194 8545 raft_consensus.cc:359] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.409297 8545 raft_consensus.cc:385] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.409329 8545 raft_consensus.cc:740] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.409474 8545 consensus_queue.cc:260] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.409511 8556 raft_consensus.cc:515] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.409889 8556 leader_election.cc:290] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.410902 8616 raft_consensus.cc:493] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.410979 8616 raft_consensus.cc:515] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.411273 8616 leader_election.cc:290] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.411473 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "773b2414ba1c46d7b6982768967d307d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:51.411578 8147 raft_consensus.cc:3060] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.411761 8545 ts_tablet_manager.cc:1434] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:51.411948 8545 tablet_bootstrap.cc:492] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.412457 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "102fc67bcf7f4336a26220fdb4f7a4d9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.412571 8146 raft_consensus.cc:2468] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.412904 8354 leader_election.cc:304] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.412948 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "773b2414ba1c46d7b6982768967d307d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:51.413065 8421 raft_consensus.cc:3060] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.413127 8616 raft_consensus.cc:2804] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.413172 8616 raft_consensus.cc:493] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.413205 8616 raft_consensus.cc:3060] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.413324 8147 raft_consensus.cc:2468] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:51.413605 8218 leader_election.cc:304] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.413784 8556 raft_consensus.cc:2804] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.413865 8556 raft_consensus.cc:697] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:51.414002 8556 consensus_queue.cc:237] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.414038 8616 raft_consensus.cc:515] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.414146 8421 raft_consensus.cc:2468] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:51.414371 8616 leader_election.cc:290] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.414741 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "102fc67bcf7f4336a26220fdb4f7a4d9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.414858 8283 raft_consensus.cc:3060] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.415125 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "102fc67bcf7f4336a26220fdb4f7a4d9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:51.415227 8147 raft_consensus.cc:3060] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.415396 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "102fc67bcf7f4336a26220fdb4f7a4d9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.415503 8545 tablet_bootstrap.cc:654] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.415877 8283 raft_consensus.cc:2468] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.416136 8551 log.cc:1199] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c5a698db1042424bad62bce674b2c44d
I20251028 02:59:51.416182 8147 raft_consensus.cc:2468] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.416466 8551 ts_tablet_manager.cc:1950] T c5a698db1042424bad62bce674b2c44d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.416555 8356 leader_election.cc:304] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:51.416807 8648 raft_consensus.cc:493] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.416882 8648 raft_consensus.cc:515] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.416927 8616 raft_consensus.cc:2804] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.417004 8616 raft_consensus.cc:697] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:51.417127 8616 consensus_queue.cc:237] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.417212 8648 leader_election.cc:290] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.417585 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4b4f077b41fc4f8b90db7392984b48a8
I20251028 02:59:51.417867 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b4f077b41fc4f8b90db7392984b48a8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.417899 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 4b4f077b41fc4f8b90db7392984b48a8 deleted and replaced by tablet 6d94d536c1054613b2ae8ffcb08fbaab
I20251028 02:59:51.417973 8283 raft_consensus.cc:2468] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.418210 8356 leader_election.cc:304] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:51.418249 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b4f077b41fc4f8b90db7392984b48a8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.418350 8147 raft_consensus.cc:2468] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.418372 8549 raft_consensus.cc:2804] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.418413 8549 raft_consensus.cc:493] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.418448 8549 raft_consensus.cc:3060] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.419638 8549 raft_consensus.cc:515] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.419991 8549 leader_election.cc:290] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.420287 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b4f077b41fc4f8b90db7392984b48a8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:51.420403 8147 raft_consensus.cc:3060] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.421289 8147 raft_consensus.cc:2468] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.421548 8354 leader_election.cc:304] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.421809 8549 raft_consensus.cc:2804] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.421896 8549 raft_consensus.cc:697] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:51.422044 8549 consensus_queue.cc:237] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.422581 8550 ts_tablet_manager.cc:1950] T 9b0abfe5364749bd86b9916480828ffd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.423350 7963 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 3e7dbafe267a4c62b0418d1b26585aad (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:51.423480 8550 tablet_replica.cc:333] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.423478 7963 catalog_manager.cc:5649] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:51.423593 8550 raft_consensus.cc:2243] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.423679 8550 raft_consensus.cc:2272] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.423717 7963 catalog_manager.cc:5649] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:51.424068 8550 ts_tablet_manager.cc:1916] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.425541 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b4f077b41fc4f8b90db7392984b48a8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.425661 8283 raft_consensus.cc:3060] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.426640 8283 raft_consensus.cc:2468] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.427444 8551 tablet_replica.cc:333] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.427598 8552 ts_tablet_manager.cc:1929] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.427678 8552 log.cc:1199] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/990a32edd54d4d85bcb6108cfb5be4d7
I20251028 02:59:51.427640 8551 raft_consensus.cc:2243] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.427881 8551 raft_consensus.cc:2272] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.427922 8552 ts_tablet_manager.cc:1950] T 990a32edd54d4d85bcb6108cfb5be4d7 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.428298 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:51.429605 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 3e7dbafe267a4c62b0418d1b26585aad (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:51.429806 8546 tablet_bootstrap.cc:492] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.429919 8546 ts_tablet_manager.cc:1403] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 02:59:51.430437 8546 raft_consensus.cc:359] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.430531 8546 raft_consensus.cc:385] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.430557 8546 raft_consensus.cc:740] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.430675 8546 consensus_queue.cc:260] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.430955 8546 ts_tablet_manager.cc:1434] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.430982 8123 tablet_service.cc:1505] Processing CreateTablet for tablet f4bebc4575e4462cbc8a02fca4fb228b (DEFAULT_TABLE table=default.Table_458c88f7cba54a43b1553e883083a60b [id=e7c4b52cbf8c496e94ca8634d53e41de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.431064 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 1c81698dcbe14d5ba61471e7249705db (DEFAULT_TABLE table=default.Table_6be0e9d6942e485798736b7f37a9bac6 [id=c103fff3458d4d08affefb7c38d35e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.431149 8546 tablet_bootstrap.cc:492] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.431203 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b4f077b41fc4f8b90db7392984b48a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 6d94d536c1054613b2ae8ffcb08fbaab) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.431277 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bebc4575e4462cbc8a02fca4fb228b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.431427 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c81698dcbe14d5ba61471e7249705db. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.432246 8546 tablet_bootstrap.cc:654] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.432498 8551 ts_tablet_manager.cc:1916] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.434101 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 (DEFAULT_TABLE table=default.Table_97bcd2a2dcfa4b08af27c1db65c4e4e2 [id=7a2a6ab3c9744942a9413c6cb057f527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.434369 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.434345 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 025a4d3e71ff443c8bdf649055a17d54 (table default.Table_7218cf040e3a418fa8c1ba0db0d4c1fe [id=7d237f6c141e4d5e821305500a020dc0]). Expected version 1 got 0
I20251028 02:59:51.434409 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 3e3cbb0c3d914ffebfd2c976ce8b1098 (DEFAULT_TABLE table=default.Table_378281b5d5ed40cda0b93ef4a3094345 [id=5dd92b20a7b94b698148c30a09b3df73]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.434573 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3cbb0c3d914ffebfd2c976ce8b1098. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.436829 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 025a4d3e71ff443c8bdf649055a17d54 (table default.Table_7218cf040e3a418fa8c1ba0db0d4c1fe [id=7d237f6c141e4d5e821305500a020dc0]). Expected version 1 got 0
I20251028 02:59:51.437883 8551 ts_tablet_manager.cc:1929] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.438045 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet fbc0337747a74e20b22aac654c2ab7cc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.438153 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9b0abfe5364749bd86b9916480828ffd (table default.Table_ef5a7a00ec9e47349f4b15f619f609f7 [id=b73cb9b72fab4a68946bb2ce1ddcf026]) successfully deleted
W20251028 02:59:51.438319 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 662c50d8fa344586bac75614ec20327b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 662c50d8fa344586bac75614ec20327b already in progress: creating tablet
I20251028 02:59:51.438499 8552 tablet_replica.cc:333] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.438634 8552 raft_consensus.cc:2243] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.439380 8551 log.cc:1199] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/773b2414ba1c46d7b6982768967d307d
I20251028 02:59:51.439378 8550 ts_tablet_manager.cc:1929] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.439260 7964 catalog_manager.cc:5649] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:51.439443 8550 log.cc:1199] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/de87fed15e1f4e41b0d962aa858e3128
I20251028 02:59:51.439531 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 8643565320504fe6b6154e936e55fca9 (table default.Table_35fd4c0e400244f28d1a855779611c46 [id=a9662d8045574f0abf81c9e6bc2f180c]). Expected version 1 got 0
I20251028 02:59:51.439667 8550 ts_tablet_manager.cc:1950] T de87fed15e1f4e41b0d962aa858e3128 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.439769 8551 ts_tablet_manager.cc:1950] T 773b2414ba1c46d7b6982768967d307d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.440443 8552 raft_consensus.cc:2272] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.441293 8546 tablet_bootstrap.cc:492] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.441293 8545 tablet_bootstrap.cc:492] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.441382 8545 ts_tablet_manager.cc:1403] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.002s
I20251028 02:59:51.441382 8546 ts_tablet_manager.cc:1403] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:51.441869 8546 raft_consensus.cc:359] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.441856 8545 raft_consensus.cc:359] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.441958 8546 raft_consensus.cc:385] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.441958 8545 raft_consensus.cc:385] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.441991 8545 raft_consensus.cc:740] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.441991 8546 raft_consensus.cc:740] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.442113 8546 consensus_queue.cc:260] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.442118 8545 consensus_queue.cc:260] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.442381 8545 ts_tablet_manager.cc:1434] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.442381 8546 ts_tablet_manager.cc:1434] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.442574 8545 tablet_bootstrap.cc:492] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.442574 8546 tablet_bootstrap.cc:492] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.443280 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet de87fed15e1f4e41b0d962aa858e3128 (table default.Table_ca53be73e3384c6ebb65c41f31a6a4b1 [id=b25233dc76034f46a52c13bb1010e305]) successfully deleted
I20251028 02:59:51.443701 8545 tablet_bootstrap.cc:654] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.445326 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b4f077b41fc4f8b90db7392984b48a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 6d94d536c1054613b2ae8ffcb08fbaab) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.445339 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2a09149ed84d4343a81458e2fb78f262"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.445616 8259 tablet_service.cc:1505] Processing CreateTablet for tablet f4bebc4575e4462cbc8a02fca4fb228b (DEFAULT_TABLE table=default.Table_458c88f7cba54a43b1553e883083a60b [id=e7c4b52cbf8c496e94ca8634d53e41de]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:51.445755 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a09149ed84d4343a81458e2fb78f262 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.445883 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bebc4575e4462cbc8a02fca4fb228b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.446826 8616 raft_consensus.cc:493] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.446907 8616 raft_consensus.cc:515] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.447285 8616 leader_election.cc:290] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.448318 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 1c81698dcbe14d5ba61471e7249705db (DEFAULT_TABLE table=default.Table_6be0e9d6942e485798736b7f37a9bac6 [id=c103fff3458d4d08affefb7c38d35e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.448429 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 3e5f0326a03040de81a506caa80dcbd9 (DEFAULT_TABLE table=default.Table_c320b509540344b7895fb967b844a408 [id=fbecd1cf849b47bb880b96a56333fbd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.448480 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b4f077b41fc4f8b90db7392984b48a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 6d94d536c1054613b2ae8ffcb08fbaab) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.448557 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c81698dcbe14d5ba61471e7249705db. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.448675 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e5f0326a03040de81a506caa80dcbd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.448738 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 1c81698dcbe14d5ba61471e7249705db (DEFAULT_TABLE table=default.Table_6be0e9d6942e485798736b7f37a9bac6 [id=c103fff3458d4d08affefb7c38d35e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.448961 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c81698dcbe14d5ba61471e7249705db. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.450686 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 (DEFAULT_TABLE table=default.Table_97bcd2a2dcfa4b08af27c1db65c4e4e2 [id=7a2a6ab3c9744942a9413c6cb057f527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.450906 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.452445 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "482d7541b350426f9ff16448a24daeed" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.452587 8147 raft_consensus.cc:2468] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.452912 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet fbc0337747a74e20b22aac654c2ab7cc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.452930 8354 leader_election.cc:304] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.453102 8616 raft_consensus.cc:2804] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.453153 8616 raft_consensus.cc:493] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.453193 8616 raft_consensus.cc:3060] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.454082 8616 raft_consensus.cc:515] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.454134 8123 tablet_service.cc:1505] Processing CreateTablet for tablet bc61d4b98fa34f8c9aa0656aa43bc25c (DEFAULT_TABLE table=default.Table_4c1aacffea564931bb49e6bc29c6b0b7 [id=449d979448934b43a49f0b431b6dc45c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.454195 8398 tablet_service.cc:1505] Processing CreateTablet for tablet f4bebc4575e4462cbc8a02fca4fb228b (DEFAULT_TABLE table=default.Table_458c88f7cba54a43b1553e883083a60b [id=e7c4b52cbf8c496e94ca8634d53e41de]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.454416 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bebc4575e4462cbc8a02fca4fb228b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.454447 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc61d4b98fa34f8c9aa0656aa43bc25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.454454 8616 leader_election.cc:290] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.454921 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "482d7541b350426f9ff16448a24daeed" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:51.455019 8147 raft_consensus.cc:3060] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.455964 8147 raft_consensus.cc:2468] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.456285 8354 leader_election.cc:304] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.456430 8616 raft_consensus.cc:2804] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.456514 8616 raft_consensus.cc:697] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:51.456634 8616 consensus_queue.cc:237] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.456662 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 (DEFAULT_TABLE table=default.Table_97bcd2a2dcfa4b08af27c1db65c4e4e2 [id=7a2a6ab3c9744942a9413c6cb057f527]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.456882 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.458899 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 3e3cbb0c3d914ffebfd2c976ce8b1098 (DEFAULT_TABLE table=default.Table_378281b5d5ed40cda0b93ef4a3094345 [id=5dd92b20a7b94b698148c30a09b3df73]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.459116 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3cbb0c3d914ffebfd2c976ce8b1098. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.459574 8551 tablet_replica.cc:333] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.459687 8551 raft_consensus.cc:2243] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.459817 8551 pending_rounds.cc:70] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:51.459847 8551 pending_rounds.cc:77] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597123266093056 op_type: NO_OP noop_request { }
I20251028 02:59:51.459898 8551 raft_consensus.cc:2889] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:51.459946 8551 raft_consensus.cc:2272] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.460950 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet de87fed15e1f4e41b0d962aa858e3128 (table default.Table_ca53be73e3384c6ebb65c41f31a6a4b1 [id=b25233dc76034f46a52c13bb1010e305]) successfully deleted
I20251028 02:59:51.461016 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet fbc0337747a74e20b22aac654c2ab7cc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:51.461090 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ab1ec08da9f64109a3fe9400fe4e366c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ab1ec08da9f64109a3fe9400fe4e366c already in progress: creating tablet
I20251028 02:59:51.461155 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet c5a698db1042424bad62bce674b2c44d (table default.Table_c1124b12dc1640bcbb89fa3d6c624c75 [id=eb9bbde4c5c54a928e854dab13c17ee5]) successfully deleted
I20251028 02:59:51.461150 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 662c50d8fa344586bac75614ec20327b with delete_type TABLET_DATA_DELETED (replaced by tablet 3e5f0326a03040de81a506caa80dcbd9) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.461215 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 773b2414ba1c46d7b6982768967d307d (table default.Table_b44dd3c16841435a8dd467a1da28db7a [id=df4041614f8a466b96efff0231711ec1]) successfully deleted
W20251028 02:59:51.461272 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ed7d32bd6b554916af9327befda800e2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ed7d32bd6b554916af9327befda800e2 already in progress: creating tablet
I20251028 02:59:51.445616 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 3e5f0326a03040de81a506caa80dcbd9 (DEFAULT_TABLE table=default.Table_c320b509540344b7895fb967b844a408 [id=fbecd1cf849b47bb880b96a56333fbd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.461418 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet cef31f29cc1a4aaa83624aac96f3da04 (table default.Table_6532259aa4cc47ee85ab998f23912c02 [id=960dc85e288d4ae6936f98938e123cf9]) successfully deleted
I20251028 02:59:51.461474 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet d09229ca82eb49f2b4214b238c3ec097 (table default.Table_fe2e8997c049473bb57e49992eeec9fa [id=7fe387bad5e740429e327d42f1efde22]) successfully deleted
I20251028 02:59:51.461522 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 990a32edd54d4d85bcb6108cfb5be4d7 (table default.Table_809e808b3698494faa7ad0258f9d5774 [id=f1e543817e8b48c8bf0e7f5ddb57330c]) successfully deleted
W20251028 02:59:51.461582 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 374936c130c841348e0d76cce070a295 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 374936c130c841348e0d76cce070a295 already in progress: creating tablet
I20251028 02:59:51.461583 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e5f0326a03040de81a506caa80dcbd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.462448 8552 ts_tablet_manager.cc:1916] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.464416 8552 ts_tablet_manager.cc:1929] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.465124 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "482d7541b350426f9ff16448a24daeed" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.465242 8283 raft_consensus.cc:2468] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.465538 8261 tablet_service.cc:1505] Processing CreateTablet for tablet bc61d4b98fa34f8c9aa0656aa43bc25c (DEFAULT_TABLE table=default.Table_4c1aacffea564931bb49e6bc29c6b0b7 [id=449d979448934b43a49f0b431b6dc45c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.465755 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc61d4b98fa34f8c9aa0656aa43bc25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.467747 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "482d7541b350426f9ff16448a24daeed" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.467891 8285 raft_consensus.cc:3060] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.468750 8285 raft_consensus.cc:2468] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.467842 8551 ts_tablet_manager.cc:1916] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.469909 7967 catalog_manager.cc:5649] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:51.471359 8544 tablet_bootstrap.cc:492] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.471474 8544 ts_tablet_manager.cc:1403] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.084s user 0.002s sys 0.000s
I20251028 02:59:51.471983 8544 raft_consensus.cc:359] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.472121 8544 raft_consensus.cc:385] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.472165 8544 raft_consensus.cc:740] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.472309 8545 tablet_bootstrap.cc:492] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.472309 8544 consensus_queue.cc:260] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.472407 8545 ts_tablet_manager.cc:1403] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20251028 02:59:51.472605 8544 ts_tablet_manager.cc:1434] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.472805 8544 tablet_bootstrap.cc:492] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.472903 8545 raft_consensus.cc:359] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.473001 8545 raft_consensus.cc:385] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.473034 8545 raft_consensus.cc:740] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.473148 8545 consensus_queue.cc:260] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.473331 8396 tablet_service.cc:1505] Processing CreateTablet for tablet bc61d4b98fa34f8c9aa0656aa43bc25c (DEFAULT_TABLE table=default.Table_4c1aacffea564931bb49e6bc29c6b0b7 [id=449d979448934b43a49f0b431b6dc45c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.473416 8545 ts_tablet_manager.cc:1434] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.473575 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc61d4b98fa34f8c9aa0656aa43bc25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.473801 8544 tablet_bootstrap.cc:654] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.474817 8545 tablet_bootstrap.cc:492] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.476076 8545 tablet_bootstrap.cc:654] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.476675 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 32ed23fd3cdc4be9ab0e65633c86e787 (DEFAULT_TABLE table=default.Table_2a09149ed84d4343a81458e2fb78f262 [id=2bcc83cc472b4ec5a7ca79a8a7c5ff23]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.476926 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32ed23fd3cdc4be9ab0e65633c86e787. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.477124 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 3e3cbb0c3d914ffebfd2c976ce8b1098 (DEFAULT_TABLE table=default.Table_378281b5d5ed40cda0b93ef4a3094345 [id=5dd92b20a7b94b698148c30a09b3df73]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.477416 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3cbb0c3d914ffebfd2c976ce8b1098. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.478571 8616 raft_consensus.cc:493] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.478655 8616 raft_consensus.cc:515] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.479020 8616 leader_election.cc:290] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.479990 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_50b58157f999483da285fc8730aba0e9" } modify_external_catalogs: true
I20251028 02:59:51.480131 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_50b58157f999483da285fc8730aba0e9" } modify_external_catalogs: true
I20251028 02:59:51.480190 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_27a9af4382b34a3eab870c4d40a6c5bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.480407 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_458c88f7cba54a43b1553e883083a60b" } new_table_name: "default.Table_69a042aded404f529721685d60c4f8b0" modify_external_catalogs: true
W20251028 02:59:51.480558 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27a9af4382b34a3eab870c4d40a6c5bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.481369 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_9c49ffa0346a4eaaabc68b637fcfc820"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.481428 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 6d94d536c1054613b2ae8ffcb08fbaab (DEFAULT_TABLE table=default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.481669 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d94d536c1054613b2ae8ffcb08fbaab. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:51.481684 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c49ffa0346a4eaaabc68b637fcfc820 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.481678 8551 ts_tablet_manager.cc:1929] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:51.481760 8551 log.cc:1199] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4b4f077b41fc4f8b90db7392984b48a8
I20251028 02:59:51.482028 8551 ts_tablet_manager.cc:1950] T 4b4f077b41fc4f8b90db7392984b48a8 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.482578 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_62299eb76b984e9ebf1136d16e827135"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.482892 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_62299eb76b984e9ebf1136d16e827135 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.482913 8552 log.cc:1199] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c5a698db1042424bad62bce674b2c44d
I20251028 02:59:51.483184 8552 ts_tablet_manager.cc:1950] T c5a698db1042424bad62bce674b2c44d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.483428 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9729143013a4ead9c3144747ecbe58c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.483429 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3adef41531ff4ba0b4f92f0bd13ad3da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.483589 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_dcbff39c1ba04e408b0a0c2ee1f42547"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.483721 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3adef41531ff4ba0b4f92f0bd13ad3da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.483807 8552 tablet_replica.cc:333] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 02:59:51.483824 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.483940 8552 raft_consensus.cc:2243] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.484120 8552 raft_consensus.cc:2272] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.484422 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2f140d5674bf4d588e435e79e52ec016"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.480000 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6532259aa4cc47ee85ab998f23912c02" } modify_external_catalogs: true
I20251028 02:59:51.484813 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6532259aa4cc47ee85ab998f23912c02" } modify_external_catalogs: true
W20251028 02:59:51.484692 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2f140d5674bf4d588e435e79e52ec016 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:51.484941 8354 leader_election.cc:343] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.485566 8551 tablet_replica.cc:333] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.485692 8551 raft_consensus.cc:2243] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.485788 8551 raft_consensus.cc:2272] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.485927 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c5a698db1042424bad62bce674b2c44d (table default.Table_c1124b12dc1640bcbb89fa3d6c624c75 [id=eb9bbde4c5c54a928e854dab13c17ee5]) successfully deleted
I20251028 02:59:51.486073 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4b4f077b41fc4f8b90db7392984b48a8 (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]) successfully deleted
I20251028 02:59:51.486274 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 6d94d536c1054613b2ae8ffcb08fbaab (DEFAULT_TABLE table=default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.486385 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9729143013a4ead9c3144747ecbe58c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:51.486219 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet fbc0337747a74e20b22aac654c2ab7cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbc0337747a74e20b22aac654c2ab7cc already in progress: creating tablet
I20251028 02:59:51.486555 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d94d536c1054613b2ae8ffcb08fbaab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.486544 8551 ts_tablet_manager.cc:1916] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.486626 8552 ts_tablet_manager.cc:1916] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.487896 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 6d94d536c1054613b2ae8ffcb08fbaab (DEFAULT_TABLE table=default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.488126 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d94d536c1054613b2ae8ffcb08fbaab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.488687 8551 ts_tablet_manager.cc:1929] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.488762 8551 log.cc:1199] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/662c50d8fa344586bac75614ec20327b
I20251028 02:59:51.488955 8551 ts_tablet_manager.cc:1950] T 662c50d8fa344586bac75614ec20327b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.489972 8545 tablet_bootstrap.cc:492] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.490057 8545 ts_tablet_manager.cc:1403] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20251028 02:59:51.490557 8545 raft_consensus.cc:359] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.490657 8545 raft_consensus.cc:385] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.490715 8545 raft_consensus.cc:740] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.490839 8545 consensus_queue.cc:260] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.491150 8545 ts_tablet_manager.cc:1434] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.491376 8545 tablet_bootstrap.cc:492] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.491573 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 32ed23fd3cdc4be9ab0e65633c86e787 (DEFAULT_TABLE table=default.Table_2a09149ed84d4343a81458e2fb78f262 [id=2bcc83cc472b4ec5a7ca79a8a7c5ff23]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.491811 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32ed23fd3cdc4be9ab0e65633c86e787. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.492404 8545 tablet_bootstrap.cc:654] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.493814 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 662c50d8fa344586bac75614ec20327b (table default.Table_c320b509540344b7895fb967b844a408 [id=fbecd1cf849b47bb880b96a56333fbd9]) successfully deleted
W20251028 02:59:51.494951 8356 leader_election.cc:343] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.495034 8356 leader_election.cc:304] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.495360 8616 raft_consensus.cc:2749] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.495787 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 32ed23fd3cdc4be9ab0e65633c86e787 (DEFAULT_TABLE table=default.Table_2a09149ed84d4343a81458e2fb78f262 [id=2bcc83cc472b4ec5a7ca79a8a7c5ff23]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.496052 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32ed23fd3cdc4be9ab0e65633c86e787. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:51.496456 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f4bebc4575e4462cbc8a02fca4fb228b consensus state has no leader
I20251028 02:59:51.498796 8546 tablet_bootstrap.cc:654] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.499607 8550 tablet_replica.cc:333] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.499801 8550 raft_consensus.cc:2243] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.499964 8550 raft_consensus.cc:2272] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.502501 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 20102243a3514ad1b5dc520b714d2413
I20251028 02:59:51.503160 8544 tablet_bootstrap.cc:492] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.503268 8544 ts_tablet_manager.cc:1403] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20251028 02:59:51.503814 8544 raft_consensus.cc:359] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.503948 8544 raft_consensus.cc:385] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.503996 8544 raft_consensus.cc:740] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.504140 8544 consensus_queue.cc:260] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.504459 8544 ts_tablet_manager.cc:1434] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.504693 8544 tablet_bootstrap.cc:492] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.505625 8544 tablet_bootstrap.cc:654] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.510298 8546 tablet_bootstrap.cc:492] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.510404 8546 ts_tablet_manager.cc:1403] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.068s user 0.001s sys 0.001s
W20251028 02:59:51.510416 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ab1ec08da9f64109a3fe9400fe4e366c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ab1ec08da9f64109a3fe9400fe4e366c already in progress: creating tablet
I20251028 02:59:51.510743 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 20102243a3514ad1b5dc520b714d2413 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.510862 8546 raft_consensus.cc:359] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.510959 8546 raft_consensus.cc:385] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.510986 8546 raft_consensus.cc:740] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.511101 8546 consensus_queue.cc:260] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.511381 8546 ts_tablet_manager.cc:1434] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.511560 8546 tablet_bootstrap.cc:492] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.513232 8544 tablet_bootstrap.cc:492] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.513332 8544 ts_tablet_manager.cc:1403] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 02:59:51.513813 8544 raft_consensus.cc:359] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.513922 8544 raft_consensus.cc:385] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.513955 8544 raft_consensus.cc:740] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.514094 8544 consensus_queue.cc:260] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.514389 8544 ts_tablet_manager.cc:1434] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.514570 8544 tablet_bootstrap.cc:492] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.516542 8546 tablet_bootstrap.cc:654] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.517305 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet bd3cebaf966245b7b009b7dd90e2eb71
I20251028 02:59:51.518445 8550 ts_tablet_manager.cc:1916] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.519330 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet bd3cebaf966245b7b009b7dd90e2eb71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.520757 8544 tablet_bootstrap.cc:654] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.522119 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet c5a698db1042424bad62bce674b2c44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.521965 8550 ts_tablet_manager.cc:1929] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.522609 8550 log.cc:1199] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/773b2414ba1c46d7b6982768967d307d
I20251028 02:59:51.522970 8546 tablet_bootstrap.cc:492] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.523065 8546 ts_tablet_manager.cc:1403] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:51.523231 8550 ts_tablet_manager.cc:1950] T 773b2414ba1c46d7b6982768967d307d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.523581 8546 raft_consensus.cc:359] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.523694 8546 raft_consensus.cc:385] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.523726 8546 raft_consensus.cc:740] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.523836 8546 consensus_queue.cc:260] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.524101 8546 ts_tablet_manager.cc:1434] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.524250 8550 tablet_replica.cc:333] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.524293 8546 tablet_bootstrap.cc:492] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.524528 8550 raft_consensus.cc:2243] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.524511 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 773b2414ba1c46d7b6982768967d307d (table default.Table_b44dd3c16841435a8dd467a1da28db7a [id=df4041614f8a466b96efff0231711ec1]) successfully deleted
W20251028 02:59:51.524696 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ed7d32bd6b554916af9327befda800e2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ed7d32bd6b554916af9327befda800e2 already in progress: creating tablet
I20251028 02:59:51.524734 8550 raft_consensus.cc:2272] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.525295 8546 tablet_bootstrap.cc:654] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.525314 8550 ts_tablet_manager.cc:1916] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.526918 8616 raft_consensus.cc:493] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.527020 8616 raft_consensus.cc:515] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.527285 8544 tablet_bootstrap.cc:492] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.527400 8544 ts_tablet_manager.cc:1403] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20251028 02:59:51.527444 8616 leader_election.cc:290] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.527765 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e7dbafe267a4c62b0418d1b26585aad" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.527895 8285 raft_consensus.cc:2468] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.527897 8544 raft_consensus.cc:359] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.527997 8544 raft_consensus.cc:385] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.528031 8544 raft_consensus.cc:740] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.528159 8544 consensus_queue.cc:260] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.528204 8356 leader_election.cc:304] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:51.528371 8616 raft_consensus.cc:2804] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.528430 8616 raft_consensus.cc:493] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.528440 8544 ts_tablet_manager.cc:1434] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.528473 8616 raft_consensus.cc:3060] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.528630 8544 tablet_bootstrap.cc:492] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.529503 8616 raft_consensus.cc:515] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.529670 8544 tablet_bootstrap.cc:654] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.529896 8616 leader_election.cc:290] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.530393 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e7dbafe267a4c62b0418d1b26585aad" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.530500 8147 raft_consensus.cc:2468] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.531035 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e7dbafe267a4c62b0418d1b26585aad" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:51.531128 8147 raft_consensus.cc:3060] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.532104 8147 raft_consensus.cc:2468] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.532708 8354 leader_election.cc:304] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.532915 8544 tablet_bootstrap.cc:492] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.532995 8544 ts_tablet_manager.cc:1403] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20251028 02:59:51.533151 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e7dbafe267a4c62b0418d1b26585aad" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.533268 8285 raft_consensus.cc:3060] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.533517 8544 raft_consensus.cc:359] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.533607 8544 raft_consensus.cc:385] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.533638 8544 raft_consensus.cc:740] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.533666 8546 tablet_bootstrap.cc:492] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.533746 8546 ts_tablet_manager.cc:1403] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 02:59:51.533749 8544 consensus_queue.cc:260] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.534022 8544 ts_tablet_manager.cc:1434] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.534188 8285 raft_consensus.cc:2468] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.534261 8546 raft_consensus.cc:359] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.534353 8546 raft_consensus.cc:385] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.534390 8546 raft_consensus.cc:740] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.534497 8546 consensus_queue.cc:260] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.534799 8546 ts_tablet_manager.cc:1434] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.534976 8546 tablet_bootstrap.cc:492] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.535740 8562 raft_consensus.cc:493] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.535836 8562 raft_consensus.cc:515] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.536265 8562 leader_election.cc:290] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.536525 8123 tablet_service.cc:1505] Processing CreateTablet for tablet a770dec97abb478b8c5504cc007b6092 (DEFAULT_TABLE table=default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.536788 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a770dec97abb478b8c5504cc007b6092. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.537027 8616 raft_consensus.cc:2804] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.537122 8616 raft_consensus.cc:697] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:51.537261 8616 consensus_queue.cc:237] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.538348 8546 tablet_bootstrap.cc:654] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.539058 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 909b08e0860b4e4f9bb1822acfd1dc02 (DEFAULT_TABLE table=default.Table_9c49ffa0346a4eaaabc68b637fcfc820 [id=98a07edd5f664674a2d33344038a8801]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.539326 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909b08e0860b4e4f9bb1822acfd1dc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.539909 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 20102243a3514ad1b5dc520b714d2413 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.541401 8122 tablet_service.cc:1505] Processing CreateTablet for tablet d928418826a84f63803ae474dc05de7e (DEFAULT_TABLE table=default.Table_27a9af4382b34a3eab870c4d40a6c5bf [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.541620 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d928418826a84f63803ae474dc05de7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.541643 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 20102243a3514ad1b5dc520b714d2413 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.542058 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 374936c130c841348e0d76cce070a295 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.541628 8550 ts_tablet_manager.cc:1929] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.542173 8550 log.cc:1199] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4b4f077b41fc4f8b90db7392984b48a8
I20251028 02:59:51.542220 8551 tablet_replica.cc:333] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.542234 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet bd3cebaf966245b7b009b7dd90e2eb71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.542351 8551 raft_consensus.cc:2243] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.542375 8550 ts_tablet_manager.cc:1950] T 4b4f077b41fc4f8b90db7392984b48a8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.542443 8551 raft_consensus.cc:2272] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.542912 8551 ts_tablet_manager.cc:1916] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.543156 8550 tablet_replica.cc:333] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.543273 8550 raft_consensus.cc:2243] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.543406 8550 raft_consensus.cc:2272] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.543707 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 71feaa313a8f45798561741f7b5d9b20 (DEFAULT_TABLE table=default.Table_62299eb76b984e9ebf1136d16e827135 [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.543931 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71feaa313a8f45798561741f7b5d9b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.544096 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 3410b1b517b146d4b38cb47862c85e68 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.544337 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3410b1b517b146d4b38cb47862c85e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.544554 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 909b08e0860b4e4f9bb1822acfd1dc02 (DEFAULT_TABLE table=default.Table_9c49ffa0346a4eaaabc68b637fcfc820 [id=98a07edd5f664674a2d33344038a8801]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.544771 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909b08e0860b4e4f9bb1822acfd1dc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.545007 8551 ts_tablet_manager.cc:1929] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.545068 8551 log.cc:1199] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/374936c130c841348e0d76cce070a295
I20251028 02:59:51.545279 8551 ts_tablet_manager.cc:1950] T 374936c130c841348e0d76cce070a295 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.546563 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 3410b1b517b146d4b38cb47862c85e68 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.546795 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3410b1b517b146d4b38cb47862c85e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.546787 8262 tablet_service.cc:1505] Processing CreateTablet for tablet a770dec97abb478b8c5504cc007b6092 (DEFAULT_TABLE table=default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.547011 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a770dec97abb478b8c5504cc007b6092. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.548686 8260 tablet_service.cc:1505] Processing CreateTablet for tablet d928418826a84f63803ae474dc05de7e (DEFAULT_TABLE table=default.Table_27a9af4382b34a3eab870c4d40a6c5bf [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.548874 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet bd3cebaf966245b7b009b7dd90e2eb71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.548921 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d928418826a84f63803ae474dc05de7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.548960 8397 tablet_service.cc:1505] Processing CreateTablet for tablet a770dec97abb478b8c5504cc007b6092 (DEFAULT_TABLE table=default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.549162 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a770dec97abb478b8c5504cc007b6092. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.549850 8544 tablet_bootstrap.cc:492] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.550958 8544 tablet_bootstrap.cc:654] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.551180 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 909b08e0860b4e4f9bb1822acfd1dc02 (DEFAULT_TABLE table=default.Table_9c49ffa0346a4eaaabc68b637fcfc820 [id=98a07edd5f664674a2d33344038a8801]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.552040 8550 ts_tablet_manager.cc:1916] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.552695 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a99123e464ae42ca9598a926c537455d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.552824 8285 raft_consensus.cc:2468] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:51.553615 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909b08e0860b4e4f9bb1822acfd1dc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.554379 8083 leader_election.cc:304] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.554546 8656 raft_consensus.cc:2804] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.554596 8656 raft_consensus.cc:493] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.554634 8656 raft_consensus.cc:3060] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.555379 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a99123e464ae42ca9598a926c537455d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:51.555482 8421 raft_consensus.cc:2468] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:51.555613 8656 raft_consensus.cc:515] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.555935 8656 leader_election.cc:290] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.556500 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a99123e464ae42ca9598a926c537455d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.556612 8285 raft_consensus.cc:3060] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.556638 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4b4f077b41fc4f8b90db7392984b48a8 (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]) successfully deleted
W20251028 02:59:51.556771 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet fbc0337747a74e20b22aac654c2ab7cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbc0337747a74e20b22aac654c2ab7cc already in progress: creating tablet
W20251028 02:59:51.556864 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 20102243a3514ad1b5dc520b714d2413 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 20102243a3514ad1b5dc520b714d2413 already in progress: creating tablet
W20251028 02:59:51.556960 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet bd3cebaf966245b7b009b7dd90e2eb71 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd3cebaf966245b7b009b7dd90e2eb71 already in progress: creating tablet
I20251028 02:59:51.557418 8285 raft_consensus.cc:2468] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:51.557662 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 3410b1b517b146d4b38cb47862c85e68 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.557749 8083 leader_election.cc:304] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.557924 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3410b1b517b146d4b38cb47862c85e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.558037 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a99123e464ae42ca9598a926c537455d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:51.558142 8421 raft_consensus.cc:3060] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.558434 8550 ts_tablet_manager.cc:1929] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.558494 8550 log.cc:1199] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c5a698db1042424bad62bce674b2c44d
I20251028 02:59:51.558694 8550 ts_tablet_manager.cc:1950] T c5a698db1042424bad62bce674b2c44d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.559130 8421 raft_consensus.cc:2468] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:51.560038 7966 catalog_manager.cc:5649] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:51.560251 7966 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 3e7dbafe267a4c62b0418d1b26585aad (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:51.561388 8656 raft_consensus.cc:2804] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.561348 8397 tablet_service.cc:1505] Processing CreateTablet for tablet d928418826a84f63803ae474dc05de7e (DEFAULT_TABLE table=default.Table_27a9af4382b34a3eab870c4d40a6c5bf [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.561484 8656 raft_consensus.cc:697] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:51.561609 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d928418826a84f63803ae474dc05de7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.561604 8656 consensus_queue.cc:237] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.563512 8616 raft_consensus.cc:493] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.563591 8616 raft_consensus.cc:515] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.563926 8616 leader_election.cc:290] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.564357 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e908d5921d34c9396c1ccdc857c66b7" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:51.564958 8354 leader_election.cc:343] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.565675 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e908d5921d34c9396c1ccdc857c66b7" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.565814 8285 raft_consensus.cc:2468] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.566110 8356 leader_election.cc:304] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.566337 8616 raft_consensus.cc:2804] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.566385 8616 raft_consensus.cc:493] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.566426 8616 raft_consensus.cc:3060] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.567358 8616 raft_consensus.cc:515] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.567399 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet c5a698db1042424bad62bce674b2c44d (table default.Table_c1124b12dc1640bcbb89fa3d6c624c75 [id=eb9bbde4c5c54a928e854dab13c17ee5]) successfully deleted
I20251028 02:59:51.567742 8616 leader_election.cc:290] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.568271 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e908d5921d34c9396c1ccdc857c66b7" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:51.568738 8354 leader_election.cc:343] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.570518 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e908d5921d34c9396c1ccdc857c66b7" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.570641 8285 raft_consensus.cc:3060] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.571696 8285 raft_consensus.cc:2468] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.572134 8356 leader_election.cc:304] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.572431 8616 raft_consensus.cc:2804] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.572530 8616 raft_consensus.cc:697] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:51.572672 8616 consensus_queue.cc:237] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.574971 7966 catalog_manager.cc:5649] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:51.575731 8562 raft_consensus.cc:493] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.575834 8562 raft_consensus.cc:515] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.576244 8562 leader_election.cc:290] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.577239 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9729143013a4ead9c3144747ecbe58c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:51.577241 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9729143013a4ead9c3144747ecbe58c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.577358 8421 raft_consensus.cc:2468] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
W20251028 02:59:51.577565 8083 leader_election.cc:343] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.577695 8083 leader_election.cc:304] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 02:59:51.577926 8562 raft_consensus.cc:2804] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.577968 8562 raft_consensus.cc:493] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.578002 8562 raft_consensus.cc:3060] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.578970 8562 raft_consensus.cc:515] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.579728 8562 leader_election.cc:290] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.580163 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9729143013a4ead9c3144747ecbe58c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 02:59:51.580488 8083 leader_election.cc:343] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.580610 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9729143013a4ead9c3144747ecbe58c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:51.580729 8421 raft_consensus.cc:3060] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.581749 8421 raft_consensus.cc:2468] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:51.582693 8083 leader_election.cc:304] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 02:59:51.582864 8546 tablet_bootstrap.cc:492] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.582909 8562 raft_consensus.cc:2804] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.582949 8546 ts_tablet_manager.cc:1403] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.000s
I20251028 02:59:51.582995 8562 raft_consensus.cc:697] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:51.583122 8562 consensus_queue.cc:237] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.583446 8546 raft_consensus.cc:359] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.583536 8546 raft_consensus.cc:385] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.583562 8546 raft_consensus.cc:740] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.583678 8546 consensus_queue.cc:260] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.583963 8546 ts_tablet_manager.cc:1434] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.584158 8546 tablet_bootstrap.cc:492] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.585186 8546 tablet_bootstrap.cc:654] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.585958 7967 catalog_manager.cc:5649] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:51.588465 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 71feaa313a8f45798561741f7b5d9b20 (DEFAULT_TABLE table=default.Table_62299eb76b984e9ebf1136d16e827135 [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.588745 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71feaa313a8f45798561741f7b5d9b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.588891 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 71feaa313a8f45798561741f7b5d9b20 (DEFAULT_TABLE table=default.Table_62299eb76b984e9ebf1136d16e827135 [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.589139 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71feaa313a8f45798561741f7b5d9b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.593621 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6be0e9d6942e485798736b7f37a9bac6" } new_table_name: "default.Table_310089f9c2b34a7e9e1c01c97fdb9b11" modify_external_catalogs: true
I20251028 02:59:51.596278 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 4726573f4b994b2591332c4a42fb10a1 (DEFAULT_TABLE table=default.Table_2f140d5674bf4d588e435e79e52ec016 [id=b0eb2947677348c9a4f7128df96afbbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.596578 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4726573f4b994b2591332c4a42fb10a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.597425 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6d18f1aded5b45cd85d34e4790ad8d40" } new_table_name: "default.Table_a28e8e2125f541a6944d94f73ee01cae" modify_external_catalogs: true
I20251028 02:59:51.599081 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 662c50d8fa344586bac75614ec20327b with delete_type TABLET_DATA_DELETED (replaced by tablet 3e5f0326a03040de81a506caa80dcbd9) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.599387 8550 tablet_replica.cc:333] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.599502 8550 raft_consensus.cc:2243] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.599581 8550 raft_consensus.cc:2272] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.600580 8283 raft_consensus.cc:1275] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:51.600589 8421 raft_consensus.cc:1275] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:51.600958 8546 tablet_bootstrap.cc:492] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.601047 8546 ts_tablet_manager.cc:1403] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 02:59:51.601006 8562 consensus_queue.cc:1048] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.601526 8546 raft_consensus.cc:359] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.601627 8546 raft_consensus.cc:385] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.601673 8546 raft_consensus.cc:740] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.601810 8546 consensus_queue.cc:260] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.602109 8546 ts_tablet_manager.cc:1434] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.602265 8546 tablet_bootstrap.cc:492] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:51.602892 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1c81698dcbe14d5ba61471e7249705db consensus state has no leader
I20251028 02:59:51.603318 8546 tablet_bootstrap.cc:654] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.603785 8550 ts_tablet_manager.cc:1916] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.606042 8550 ts_tablet_manager.cc:1929] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.606128 8550 log.cc:1199] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/662c50d8fa344586bac75614ec20327b
I20251028 02:59:51.606177 8562 consensus_queue.cc:1048] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.606338 8550 ts_tablet_manager.cc:1950] T 662c50d8fa344586bac75614ec20327b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.607245 8546 tablet_bootstrap.cc:492] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.607355 8546 ts_tablet_manager.cc:1403] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:51.607504 7967 catalog_manager.cc:5649] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:51.607851 8546 raft_consensus.cc:359] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.607966 8546 raft_consensus.cc:385] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.608003 8546 raft_consensus.cc:740] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.608140 8546 consensus_queue.cc:260] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.608443 8546 ts_tablet_manager.cc:1434] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:51.608628 8546 tablet_bootstrap.cc:492] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.608953 8553 raft_consensus.cc:493] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.609030 8553 raft_consensus.cc:515] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.609386 8553 leader_election.cc:290] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.609648 8546 tablet_bootstrap.cc:654] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.609879 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "025a4d3e71ff443c8bdf649055a17d54" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:51.610283 8220 leader_election.cc:343] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.610414 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 8643565320504fe6b6154e936e55fca9 (table default.Table_35fd4c0e400244f28d1a855779611c46 [id=a9662d8045574f0abf81c9e6bc2f180c]). Expected version 1 got 0
I20251028 02:59:51.610706 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "025a4d3e71ff443c8bdf649055a17d54" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:51.611033 8218 leader_election.cc:343] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.611111 8218 leader_election.cc:304] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.611327 8556 raft_consensus.cc:2749] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.611634 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c320b509540344b7895fb967b844a408" } modify_external_catalogs: true
I20251028 02:59:51.611747 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c320b509540344b7895fb967b844a408" } modify_external_catalogs: true
I20251028 02:59:51.612210 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_49d0ca6e0aa54b309562cc456bcf1be5" } new_table_name: "default.Table_94d71cfa43134dff9a62343374aed8fe" modify_external_catalogs: true
I20251028 02:59:51.612433 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 8643565320504fe6b6154e936e55fca9 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.613735 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 6c6c40605a5a48d79837893443f4ca1b from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.616407 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_eba1e050b6d54c418fbfec1ffcfc8052" } modify_external_catalogs: true
I20251028 02:59:51.616513 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_eba1e050b6d54c418fbfec1ffcfc8052" } modify_external_catalogs: true
I20251028 02:59:51.616986 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.617329 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.617378 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8643565320504fe6b6154e936e55fca9
I20251028 02:59:51.617645 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 8643565320504fe6b6154e936e55fca9 deleted and replaced by tablet 8cb97d5192684d748b8b485f891b0dc0
I20251028 02:59:51.620059 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 662c50d8fa344586bac75614ec20327b (table default.Table_c320b509540344b7895fb967b844a408 [id=fbecd1cf849b47bb880b96a56333fbd9]) successfully deleted
I20251028 02:59:51.620043 8726 tablet.cc:1722] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.621091 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 8643565320504fe6b6154e936e55fca9 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cb97d5192684d748b8b485f891b0dc0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.621661 8782 tablet.cc:1722] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.623404 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6c6c40605a5a48d79837893443f4ca1b
I20251028 02:59:51.623683 7964 catalog_manager.cc:6719] ReplaceTablet: tablet 6c6c40605a5a48d79837893443f4ca1b deleted and replaced by tablet aa1fe079ad014fe0b0d99efe7981de88
I20251028 02:59:51.625279 8550 tablet_replica.cc:333] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.625422 8550 raft_consensus.cc:2243] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.625581 8550 raft_consensus.cc:2272] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.625696 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c6c40605a5a48d79837893443f4ca1b with delete_type TABLET_DATA_DELETED (replaced by tablet aa1fe079ad014fe0b0d99efe7981de88) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.626466 8550 ts_tablet_manager.cc:1916] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.628919 8550 ts_tablet_manager.cc:1929] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:51.629036 8550 log.cc:1199] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8643565320504fe6b6154e936e55fca9
I20251028 02:59:51.630842 8550 ts_tablet_manager.cc:1950] T 8643565320504fe6b6154e936e55fca9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.633455 8546 tablet_bootstrap.cc:492] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.633554 8546 ts_tablet_manager.cc:1403] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20251028 02:59:51.633955 8546 raft_consensus.cc:359] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.634063 8546 raft_consensus.cc:385] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.634106 8546 raft_consensus.cc:740] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.634229 8546 consensus_queue.cc:260] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.634476 8546 ts_tablet_manager.cc:1434] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.634649 8546 tablet_bootstrap.cc:492] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.635473 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8643565320504fe6b6154e936e55fca9 (table default.Table_35fd4c0e400244f28d1a855779611c46 [id=a9662d8045574f0abf81c9e6bc2f180c]) successfully deleted
W20251028 02:59:51.635655 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6c6c40605a5a48d79837893443f4ca1b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6c6c40605a5a48d79837893443f4ca1b already in progress: creating tablet
I20251028 02:59:51.636886 8552 ts_tablet_manager.cc:1929] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.636973 8552 log.cc:1199] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/662c50d8fa344586bac75614ec20327b
I20251028 02:59:51.637223 8552 ts_tablet_manager.cc:1950] T 662c50d8fa344586bac75614ec20327b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.637638 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_620185a247324af6b2054263457ec7da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.637812 8546 tablet_bootstrap.cc:654] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:51.638229 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_620185a247324af6b2054263457ec7da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.639927 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 909b08e0860b4e4f9bb1822acfd1dc02 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.640272 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet f11c46ed489e41d3a25cfcd9413443af from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.640295 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 66fb0173e40e40999f90f015ddb294b7
I20251028 02:59:51.640328 8552 tablet_replica.cc:333] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.640463 8552 raft_consensus.cc:2243] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.640559 8556 raft_consensus.cc:493] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.640610 8552 raft_consensus.cc:2272] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.640645 8556 raft_consensus.cc:515] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:51.640714 7955 catalog_manager.cc:5058] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): alter failed for tablet 0876f8b4ee634d9d92efe9613b3495be (table default.Table_97bcd2a2dcfa4b08af27c1db65c4e4e2 [id=7a2a6ab3c9744942a9413c6cb057f527]),no further retry: Not found: Tablet not found: 0876f8b4ee634d9d92efe9613b3495be
I20251028 02:59:51.640972 8552 ts_tablet_manager.cc:1916] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.641005 8556 leader_election.cc:290] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.641364 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 66fb0173e40e40999f90f015ddb294b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.641700 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f67b25b85194ce0be43f01adccd6c48" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:51.642139 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f67b25b85194ce0be43f01adccd6c48" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:51.642809 8220 leader_election.cc:343] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:51.642987 8218 leader_election.cc:343] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.643040 8218 leader_election.cc:304] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.643097 8552 ts_tablet_manager.cc:1929] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.643167 8552 log.cc:1199] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/de87fed15e1f4e41b0d962aa858e3128
I20251028 02:59:51.643393 8552 ts_tablet_manager.cc:1950] T de87fed15e1f4e41b0d962aa858e3128 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.643409 8556 raft_consensus.cc:2749] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.648036 8552 tablet_replica.cc:333] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.649055 8546 tablet_bootstrap.cc:492] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.649190 8546 ts_tablet_manager.cc:1403] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 02:59:51.649600 8546 raft_consensus.cc:359] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.649698 8546 raft_consensus.cc:385] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.649724 8546 raft_consensus.cc:740] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.649848 8546 consensus_queue.cc:260] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.650113 8546 ts_tablet_manager.cc:1434] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:51.650312 8546 tablet_bootstrap.cc:492] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.651262 8546 tablet_bootstrap.cc:654] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.652383 8552 raft_consensus.cc:2243] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.652540 8552 pending_rounds.cc:70] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:51.652567 8552 pending_rounds.cc:77] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597123250368512 op_type: NO_OP noop_request { }
I20251028 02:59:51.652613 8552 raft_consensus.cc:2889] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:51.652657 8552 raft_consensus.cc:2272] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.653167 8552 ts_tablet_manager.cc:1916] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:51.640865 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 20102243a3514ad1b5dc520b714d2413 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 20102243a3514ad1b5dc520b714d2413 already in progress: creating tablet
I20251028 02:59:51.654501 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 374936c130c841348e0d76cce070a295 (table default.Table_bb2a8beb93a74d8eb05afe4fb154bbfa [id=90264f1fa3994b8a944e512a95361ff2]) successfully deleted
W20251028 02:59:51.654783 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet bd3cebaf966245b7b009b7dd90e2eb71 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd3cebaf966245b7b009b7dd90e2eb71 already in progress: creating tablet
I20251028 02:59:51.651283 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f11c46ed489e41d3a25cfcd9413443af
I20251028 02:59:51.655514 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 4726573f4b994b2591332c4a42fb10a1 (DEFAULT_TABLE table=default.Table_2f140d5674bf4d588e435e79e52ec016 [id=b0eb2947677348c9a4f7128df96afbbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.655712 7969 catalog_manager.cc:6719] ReplaceTablet: tablet f11c46ed489e41d3a25cfcd9413443af deleted and replaced by tablet 9f1c1e14665c4712bffa439ab98b2a6e
I20251028 02:59:51.655836 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4726573f4b994b2591332c4a42fb10a1. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:51.652287 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 66fb0173e40e40999f90f015ddb294b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66fb0173e40e40999f90f015ddb294b7 already in progress: creating tablet
I20251028 02:59:51.656284 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet f11c46ed489e41d3a25cfcd9413443af with delete_type TABLET_DATA_DELETED (replaced by tablet 9f1c1e14665c4712bffa439ab98b2a6e) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.656970 8552 ts_tablet_manager.cc:1929] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:51.657037 8552 log.cc:1199] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/773b2414ba1c46d7b6982768967d307d
I20251028 02:59:51.657202 8544 tablet_bootstrap.cc:492] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.657303 8544 ts_tablet_manager.cc:1403] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.108s user 0.000s sys 0.002s
I20251028 02:59:51.657344 8552 ts_tablet_manager.cc:1950] T 773b2414ba1c46d7b6982768967d307d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.657792 8544 raft_consensus.cc:359] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.657931 8544 raft_consensus.cc:385] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.657968 8544 raft_consensus.cc:740] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.658114 8544 consensus_queue.cc:260] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.658335 8545 tablet_bootstrap.cc:492] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.658398 8544 ts_tablet_manager.cc:1434] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.658414 8545 ts_tablet_manager.cc:1403] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.167s user 0.002s sys 0.000s
I20251028 02:59:51.658617 8544 tablet_bootstrap.cc:492] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.658900 8545 raft_consensus.cc:359] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:51.658953 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f11c46ed489e41d3a25cfcd9413443af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f11c46ed489e41d3a25cfcd9413443af already in progress: creating tablet
I20251028 02:59:51.658995 8545 raft_consensus.cc:385] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.659035 8545 raft_consensus.cc:740] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.659157 8545 consensus_queue.cc:260] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.659451 8545 ts_tablet_manager.cc:1434] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.659641 8545 tablet_bootstrap.cc:492] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.641651 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ab9ea7fcee93462394ebd14c2006d4e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.660645 8545 tablet_bootstrap.cc:654] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.663175 8552 tablet_replica.cc:333] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.663379 8552 raft_consensus.cc:2243] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.663483 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 20102243a3514ad1b5dc520b714d2413 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.663518 8552 raft_consensus.cc:2272] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.663980 8552 ts_tablet_manager.cc:1916] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:51.664036 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab9ea7fcee93462394ebd14c2006d4e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.666033 8552 ts_tablet_manager.cc:1929] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.666152 8552 log.cc:1199] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/ed7d32bd6b554916af9327befda800e2
I20251028 02:59:51.666436 8552 ts_tablet_manager.cc:1950] T ed7d32bd6b554916af9327befda800e2 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.667110 8552 tablet_replica.cc:333] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.667200 8545 tablet_bootstrap.cc:492] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.667311 8545 ts_tablet_manager.cc:1403] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:51.667459 8552 raft_consensus.cc:2243] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.667671 8552 raft_consensus.cc:2272] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.668026 8545 raft_consensus.cc:359] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.668131 8544 tablet_bootstrap.cc:654] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.668133 8545 raft_consensus.cc:385] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.668171 8545 raft_consensus.cc:740] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.668284 8545 consensus_queue.cc:260] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.668536 8545 ts_tablet_manager.cc:1434] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.668731 8545 tablet_bootstrap.cc:492] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.669781 8545 tablet_bootstrap.cc:654] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.671180 8545 tablet_bootstrap.cc:492] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.671263 8545 ts_tablet_manager.cc:1403] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:51.671751 8545 raft_consensus.cc:359] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.671868 8545 raft_consensus.cc:385] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.671904 8545 raft_consensus.cc:740] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.672036 8545 consensus_queue.cc:260] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.672253 8546 tablet_bootstrap.cc:492] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.672297 8545 ts_tablet_manager.cc:1434] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.672335 8546 ts_tablet_manager.cc:1403] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20251028 02:59:51.672454 8545 tablet_bootstrap.cc:492] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.672771 8546 raft_consensus.cc:359] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.672879 8546 raft_consensus.cc:385] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.672915 8546 raft_consensus.cc:740] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.673046 8546 consensus_queue.cc:260] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.673307 8546 ts_tablet_manager.cc:1434] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.673439 8545 tablet_bootstrap.cc:654] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.673519 8546 tablet_bootstrap.cc:492] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.674053 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 025a4d3e71ff443c8bdf649055a17d54 (table default.Table_7218cf040e3a418fa8c1ba0db0d4c1fe [id=7d237f6c141e4d5e821305500a020dc0]). Expected version 1 got 0
I20251028 02:59:51.674409 8546 tablet_bootstrap.cc:654] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.675642 8545 tablet_bootstrap.cc:492] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.675712 8545 ts_tablet_manager.cc:1403] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:51.676185 8545 raft_consensus.cc:359] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.676285 8545 raft_consensus.cc:385] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.676322 8545 raft_consensus.cc:740] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.676434 8545 consensus_queue.cc:260] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.676678 8545 ts_tablet_manager.cc:1434] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.676847 8545 tablet_bootstrap.cc:492] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.677804 8545 tablet_bootstrap.cc:654] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.678043 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 909b08e0860b4e4f9bb1822acfd1dc02
I20251028 02:59:51.678315 7961 catalog_manager.cc:6719] ReplaceTablet: tablet 909b08e0860b4e4f9bb1822acfd1dc02 deleted and replaced by tablet 8b88fda3f40540f3b42ee7f33dd33251
I20251028 02:59:51.679046 8552 ts_tablet_manager.cc:1916] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.679615 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 025a4d3e71ff443c8bdf649055a17d54 (table default.Table_7218cf040e3a418fa8c1ba0db0d4c1fe [id=7d237f6c141e4d5e821305500a020dc0]). Expected version 1 got 0
I20251028 02:59:51.681324 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 909b08e0860b4e4f9bb1822acfd1dc02 with delete_type TABLET_DATA_DELETED (replaced by tablet 8b88fda3f40540f3b42ee7f33dd33251) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.681381 8545 tablet_bootstrap.cc:492] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.681465 8545 ts_tablet_manager.cc:1403] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:51.682016 8545 raft_consensus.cc:359] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.682124 8545 raft_consensus.cc:385] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.682166 8545 raft_consensus.cc:740] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.682292 8545 consensus_queue.cc:260] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.682444 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 8cb97d5192684d748b8b485f891b0dc0 (DEFAULT_TABLE table=default.Table_35fd4c0e400244f28d1a855779611c46 [id=a9662d8045574f0abf81c9e6bc2f180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.682570 8545 ts_tablet_manager.cc:1434] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.682727 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cb97d5192684d748b8b485f891b0dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.682761 8545 tablet_bootstrap.cc:492] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.682986 8552 ts_tablet_manager.cc:1929] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.683053 8552 log.cc:1199] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4b4f077b41fc4f8b90db7392984b48a8
I20251028 02:59:51.683283 8552 ts_tablet_manager.cc:1950] T 4b4f077b41fc4f8b90db7392984b48a8 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.683957 8545 tablet_bootstrap.cc:654] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.684474 8546 tablet_bootstrap.cc:492] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.684558 8546 ts_tablet_manager.cc:1403] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20251028 02:59:51.685006 8545 tablet_bootstrap.cc:492] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.685091 8545 ts_tablet_manager.cc:1403] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:51.685012 8546 raft_consensus.cc:359] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.685132 8546 raft_consensus.cc:385] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.685178 8546 raft_consensus.cc:740] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.685312 8546 consensus_queue.cc:260] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.685452 8656 raft_consensus.cc:493] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.685539 8656 raft_consensus.cc:515] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.685531 8545 raft_consensus.cc:359] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.685598 8546 ts_tablet_manager.cc:1434] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.685624 8545 raft_consensus.cc:385] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.685655 8545 raft_consensus.cc:740] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.685775 8546 tablet_bootstrap.cc:492] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.685765 8545 consensus_queue.cc:260] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.685920 8656 leader_election.cc:290] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.686038 8545 ts_tablet_manager.cc:1434] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.686210 8545 tablet_bootstrap.cc:492] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.686735 8546 tablet_bootstrap.cc:654] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.686684 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3209160f7b0e4f1a8c8121725454d088" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:51.687166 8545 tablet_bootstrap.cc:654] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.687721 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3209160f7b0e4f1a8c8121725454d088" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.687865 8283 raft_consensus.cc:2393] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 02:59:51.688401 8083 leader_election.cc:304] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:51.688616 8552 tablet_replica.cc:333] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.688650 8656 raft_consensus.cc:3060] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.688838 8552 raft_consensus.cc:2243] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.688953 8552 raft_consensus.cc:2272] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.689236 8545 tablet_bootstrap.cc:492] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.689311 8545 ts_tablet_manager.cc:1403] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251028 02:59:51.689414 8552 ts_tablet_manager.cc:1916] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.689640 8656 raft_consensus.cc:2749] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.689797 8545 raft_consensus.cc:359] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.689917 8545 raft_consensus.cc:385] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.689952 8545 raft_consensus.cc:740] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.690057 8545 consensus_queue.cc:260] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.690328 8545 ts_tablet_manager.cc:1434] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.690498 8545 tablet_bootstrap.cc:492] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.691486 8545 tablet_bootstrap.cc:654] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.692260 8546 tablet_bootstrap.cc:492] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.692346 8546 ts_tablet_manager.cc:1403] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:51.692824 8546 raft_consensus.cc:359] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.692929 8546 raft_consensus.cc:385] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.692960 8546 raft_consensus.cc:740] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.693078 8546 consensus_queue.cc:260] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.693326 8546 ts_tablet_manager.cc:1434] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.694415 8552 ts_tablet_manager.cc:1929] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.694528 8552 log.cc:1199] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/20102243a3514ad1b5dc520b714d2413
I20251028 02:59:51.694777 8552 ts_tablet_manager.cc:1950] T 20102243a3514ad1b5dc520b714d2413 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.694988 8546 tablet_bootstrap.cc:492] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.695989 8546 tablet_bootstrap.cc:654] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.697145 8552 tablet_replica.cc:333] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.697405 8552 raft_consensus.cc:2243] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.697544 8552 raft_consensus.cc:2272] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.698000 8552 ts_tablet_manager.cc:1916] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.698354 8122 tablet_service.cc:1505] Processing CreateTablet for tablet d248313db35e40e38e2c5e07e68a69af (DEFAULT_TABLE table=default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.698678 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d248313db35e40e38e2c5e07e68a69af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.700235 8552 ts_tablet_manager.cc:1929] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.700352 8552 log.cc:1199] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/bd3cebaf966245b7b009b7dd90e2eb71
I20251028 02:59:51.700603 8552 ts_tablet_manager.cc:1950] T bd3cebaf966245b7b009b7dd90e2eb71 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.700906 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 11a8056e97ff452e93e91622fedb77a6 (table default.Table_fe2e8997c049473bb57e49992eeec9fa [id=7fe387bad5e740429e327d42f1efde22]). Expected version 3 got 0
I20251028 02:59:51.701699 8546 tablet_bootstrap.cc:492] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.701774 8546 ts_tablet_manager.cc:1403] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:51.702209 8546 raft_consensus.cc:359] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.702234 8122 tablet_service.cc:1505] Processing CreateTablet for tablet cfd68d91fd9b4c69871383e90bd2ff85 (DEFAULT_TABLE table=default.Table_620185a247324af6b2054263457ec7da [id=c33a0072d1384953b73c2e3d90176cc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.702315 8546 raft_consensus.cc:385] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.702351 8546 raft_consensus.cc:740] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.702481 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd68d91fd9b4c69871383e90bd2ff85. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.702458 8546 consensus_queue.cc:260] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.702699 8546 ts_tablet_manager.cc:1434] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.702858 8546 tablet_bootstrap.cc:492] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.703815 8546 tablet_bootstrap.cc:654] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.704782 8546 tablet_bootstrap.cc:492] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.704849 8546 ts_tablet_manager.cc:1403] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20251028 02:59:51.705245 8546 raft_consensus.cc:359] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.705341 8546 raft_consensus.cc:385] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.705379 8546 raft_consensus.cc:740] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.705487 8546 consensus_queue.cc:260] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.705753 8546 ts_tablet_manager.cc:1434] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.705897 8546 tablet_bootstrap.cc:492] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.707007 8546 tablet_bootstrap.cc:654] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.709283 8546 tablet_bootstrap.cc:492] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.709367 8546 ts_tablet_manager.cc:1403] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251028 02:59:51.709554 8545 tablet_bootstrap.cc:492] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.709640 8545 ts_tablet_manager.cc:1403] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20251028 02:59:51.709702 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3e5f0326a03040de81a506caa80dcbd9
I20251028 02:59:51.709892 8546 raft_consensus.cc:359] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.709998 8546 raft_consensus.cc:385] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.710037 8546 raft_consensus.cc:740] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.710157 8546 consensus_queue.cc:260] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.710155 8545 raft_consensus.cc:359] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.710260 8545 raft_consensus.cc:385] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.710299 8545 raft_consensus.cc:740] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.710404 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e5f0326a03040de81a506caa80dcbd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.710451 8546 ts_tablet_manager.cc:1434] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.710429 8545 consensus_queue.cc:260] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.710605 8546 tablet_bootstrap.cc:492] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.710691 8545 ts_tablet_manager.cc:1434] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.710873 8545 tablet_bootstrap.cc:492] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.711673 8546 tablet_bootstrap.cc:654] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.711922 8545 tablet_bootstrap.cc:654] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.713799 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 9f1c1e14665c4712bffa439ab98b2a6e (DEFAULT_TABLE table=default.Table_ec3580a0e3af454f9154d6733e07ca55 [id=3eb0b70129be40e598874d492735ba51]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.714139 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f1c1e14665c4712bffa439ab98b2a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.715003 8546 tablet_bootstrap.cc:492] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.715097 8546 ts_tablet_manager.cc:1403] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20251028 02:59:51.715559 8546 raft_consensus.cc:359] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.715659 8546 raft_consensus.cc:385] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.715691 8546 raft_consensus.cc:740] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.715801 8546 consensus_queue.cc:260] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.716086 8546 ts_tablet_manager.cc:1434] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.716271 8546 tablet_bootstrap.cc:492] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.717183 8546 tablet_bootstrap.cc:654] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.717433 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 11a8056e97ff452e93e91622fedb77a6 (table default.Table_fe2e8997c049473bb57e49992eeec9fa [id=7fe387bad5e740429e327d42f1efde22]). Expected version 3 got 0
I20251028 02:59:51.720919 8553 raft_consensus.cc:493] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.721017 8553 raft_consensus.cc:515] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.721391 8553 leader_election.cc:290] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:51.721493 8550 tablet_replica.cc:333] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.721626 8550 raft_consensus.cc:2243] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.721737 8550 raft_consensus.cc:2272] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.721839 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f11c46ed489e41d3a25cfcd9413443af" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:51.722246 8218 leader_election.cc:343] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.722292 8550 ts_tablet_manager.cc:1916] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.723448 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f11c46ed489e41d3a25cfcd9413443af" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:51.723886 8220 leader_election.cc:343] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.723960 8220 leader_election.cc:304] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.724198 8553 raft_consensus.cc:2749] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.725695 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 4726573f4b994b2591332c4a42fb10a1 (DEFAULT_TABLE table=default.Table_2f140d5674bf4d588e435e79e52ec016 [id=b0eb2947677348c9a4f7128df96afbbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.727070 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 8643565320504fe6b6154e936e55fca9 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cb97d5192684d748b8b485f891b0dc0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.727237 8551 tablet_replica.cc:333] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.726337 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 8643565320504fe6b6154e936e55fca9 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cb97d5192684d748b8b485f891b0dc0) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.727397 8551 raft_consensus.cc:2243] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.727417 8552 tablet_replica.cc:333] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.727479 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4726573f4b994b2591332c4a42fb10a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.727491 8551 pending_rounds.cc:70] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 2 pending ops.
I20251028 02:59:51.727558 8552 raft_consensus.cc:2243] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.727517 8551 pending_rounds.cc:77] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7215597123990061056 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "8643565320504fe6b6154e936e55fca9" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_35fd4c0e400244f28d1a855779611c46" dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" new_extra_config { } }
I20251028 02:59:51.727674 8552 raft_consensus.cc:2272] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.727741 8551 pending_rounds.cc:77] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597123212173312 op_type: NO_OP noop_request { }
I20251028 02:59:51.727795 8551 raft_consensus.cc:2889] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:51.727860 8551 raft_consensus.cc:2272] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.726975 8283 raft_consensus.cc:1275] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:51.728472 8552 ts_tablet_manager.cc:1916] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:51.728603 8354 consensus_peers.cc:597] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 -> Peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Couldn't send request to peer fbd2afb4894c4e7a8520cc09f8593aa0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:51.730212 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 11a8056e97ff452e93e91622fedb77a6 (table default.Table_fe2e8997c049473bb57e49992eeec9fa [id=7fe387bad5e740429e327d42f1efde22]). Expected version 3 got 0
I20251028 02:59:51.730434 8552 ts_tablet_manager.cc:1929] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:51.730494 8552 log.cc:1199] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/8643565320504fe6b6154e936e55fca9
I20251028 02:59:51.730722 8552 ts_tablet_manager.cc:1950] T 8643565320504fe6b6154e936e55fca9 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.731458 8549 consensus_queue.cc:1048] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.732517 8550 ts_tablet_manager.cc:1929] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.732488 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 8b88fda3f40540f3b42ee7f33dd33251 (DEFAULT_TABLE table=default.Table_9c49ffa0346a4eaaabc68b637fcfc820 [id=98a07edd5f664674a2d33344038a8801]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.732586 8550 log.cc:1199] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/20102243a3514ad1b5dc520b714d2413
I20251028 02:59:51.732754 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b88fda3f40540f3b42ee7f33dd33251. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.732815 8550 ts_tablet_manager.cc:1950] T 20102243a3514ad1b5dc520b714d2413 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.733433 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 20102243a3514ad1b5dc520b714d2413 (table default.Table_50b58157f999483da285fc8730aba0e9 [id=3f05bbb6510e4149acbb9d0c236ee4f0]) successfully deleted
W20251028 02:59:51.733752 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 909b08e0860b4e4f9bb1822acfd1dc02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 909b08e0860b4e4f9bb1822acfd1dc02 already in progress: creating tablet
W20251028 02:59:51.734083 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 3e5f0326a03040de81a506caa80dcbd9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3e5f0326a03040de81a506caa80dcbd9 already in progress: creating tablet
I20251028 02:59:51.735352 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c6c40605a5a48d79837893443f4ca1b with delete_type TABLET_DATA_DELETED (replaced by tablet aa1fe079ad014fe0b0d99efe7981de88) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.737027 8147 raft_consensus.cc:1275] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:51.737030 8285 raft_consensus.cc:1275] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:51.737208 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 0868c69e28a840b4ab9dad61d50983ec (DEFAULT_TABLE table=default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.737407 8616 consensus_queue.cc:1048] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.737495 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0868c69e28a840b4ab9dad61d50983ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.737682 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 9e908d5921d34c9396c1ccdc857c66b7 (table default.Table_a28e8e2125f541a6944d94f73ee01cae [id=4eb2743632814baab2e39bbdce700d6b]). Expected version 1 got 0
I20251028 02:59:51.738155 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 66fb0173e40e40999f90f015ddb294b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.738269 8616 consensus_queue.cc:1048] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.738996 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 66fb0173e40e40999f90f015ddb294b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.740089 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet de87fed15e1f4e41b0d962aa858e3128 (table default.Table_ca53be73e3384c6ebb65c41f31a6a4b1 [id=b25233dc76034f46a52c13bb1010e305]) successfully deleted
I20251028 02:59:51.740928 8546 tablet_bootstrap.cc:492] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.741009 8546 ts_tablet_manager.cc:1403] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20251028 02:59:51.741593 8546 raft_consensus.cc:359] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.741695 8546 raft_consensus.cc:385] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.741721 8546 raft_consensus.cc:740] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.741824 8546 consensus_queue.cc:260] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.742084 8546 ts_tablet_manager.cc:1434] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.742224 8546 tablet_bootstrap.cc:492] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.743211 8546 tablet_bootstrap.cc:654] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:51.740219 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ab1ec08da9f64109a3fe9400fe4e366c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ab1ec08da9f64109a3fe9400fe4e366c already in progress: creating tablet
I20251028 02:59:51.744021 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 773b2414ba1c46d7b6982768967d307d (table default.Table_b44dd3c16841435a8dd467a1da28db7a [id=df4041614f8a466b96efff0231711ec1]) successfully deleted
I20251028 02:59:51.744136 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet ed7d32bd6b554916af9327befda800e2 (table default.Table_310089f9c2b34a7e9e1c01c97fdb9b11 [id=c103fff3458d4d08affefb7c38d35e52]) successfully deleted
I20251028 02:59:51.744212 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4b4f077b41fc4f8b90db7392984b48a8 (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]) successfully deleted
W20251028 02:59:51.744309 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet fbc0337747a74e20b22aac654c2ab7cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbc0337747a74e20b22aac654c2ab7cc already in progress: creating tablet
I20251028 02:59:51.744391 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 20102243a3514ad1b5dc520b714d2413 (table default.Table_50b58157f999483da285fc8730aba0e9 [id=3f05bbb6510e4149acbb9d0c236ee4f0]) successfully deleted
I20251028 02:59:51.744472 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet bd3cebaf966245b7b009b7dd90e2eb71 (table default.Table_6532259aa4cc47ee85ab998f23912c02 [id=960dc85e288d4ae6936f98938e123cf9]) successfully deleted
I20251028 02:59:51.744593 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 8643565320504fe6b6154e936e55fca9 (table default.Table_35fd4c0e400244f28d1a855779611c46 [id=a9662d8045574f0abf81c9e6bc2f180c]) successfully deleted
W20251028 02:59:51.744678 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 66fb0173e40e40999f90f015ddb294b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66fb0173e40e40999f90f015ddb294b7 already in progress: creating tablet
I20251028 02:59:51.745014 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_9c49ffa0346a4eaaabc68b637fcfc820" } new_table_name: "default.Table_7fb744c8c91e4c9daadacdab1a408fa1" modify_external_catalogs: true
I20251028 02:59:51.745149 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_fc88d863fb634dc59631a752b9afe967"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.745261 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_78628adeb83d46c1913ffa57d5336dcc" } modify_external_catalogs: true
I20251028 02:59:51.745332 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_78628adeb83d46c1913ffa57d5336dcc" } modify_external_catalogs: true
I20251028 02:59:51.745352 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet f11c46ed489e41d3a25cfcd9413443af with delete_type TABLET_DATA_DELETED (replaced by tablet 9f1c1e14665c4712bffa439ab98b2a6e) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:51.745601 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc88d863fb634dc59631a752b9afe967 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.745023 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet c5971fbd47e64ffda6df61dfc4756dfb from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.746131 8544 tablet_bootstrap.cc:492] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.746227 8544 ts_tablet_manager.cc:1403] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.088s user 0.002s sys 0.000s
I20251028 02:59:51.746726 8544 raft_consensus.cc:359] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.746842 8544 raft_consensus.cc:385] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.746950 8544 raft_consensus.cc:740] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.747125 8544 consensus_queue.cc:260] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.748035 8544 ts_tablet_manager.cc:1434] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:51.748211 8544 tablet_bootstrap.cc:492] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.749186 8544 tablet_bootstrap.cc:654] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.750998 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet f11c46ed489e41d3a25cfcd9413443af with delete_type TABLET_DATA_DELETED (replaced by tablet 9f1c1e14665c4712bffa439ab98b2a6e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.751644 8545 tablet_bootstrap.cc:492] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.751657 8552 tablet_replica.cc:333] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.751749 8545 ts_tablet_manager.cc:1403] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.041s user 0.000s sys 0.002s
I20251028 02:59:51.751801 8552 raft_consensus.cc:2243] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.751924 8552 raft_consensus.cc:2272] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.752250 8545 raft_consensus.cc:359] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.752357 8545 raft_consensus.cc:385] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.752389 8545 raft_consensus.cc:740] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.752511 8545 consensus_queue.cc:260] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.752663 8552 ts_tablet_manager.cc:1916] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.752810 8545 ts_tablet_manager.cc:1434] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.752995 8545 tablet_bootstrap.cc:492] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:51.753888 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b88fda3f40540f3b42ee7f33dd33251 consensus state has no leader
I20251028 02:59:51.753865 8753 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.754011 8545 tablet_bootstrap.cc:654] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.755059 8552 ts_tablet_manager.cc:1929] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.755124 8552 log.cc:1199] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/f11c46ed489e41d3a25cfcd9413443af
I20251028 02:59:51.755270 8782 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.755558 8553 raft_consensus.cc:493] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.755630 8553 raft_consensus.cc:515] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.755683 8552 ts_tablet_manager.cc:1950] T f11c46ed489e41d3a25cfcd9413443af P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.755981 8553 leader_election.cc:290] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.756772 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "374936c130c841348e0d76cce070a295" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:51.757014 8220 leader_election.cc:343] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 374936c130c841348e0d76cce070a295
W20251028 02:59:51.757303 8218 leader_election.cc:343] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.757369 8218 leader_election.cc:304] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:51.757398 8848 consensus_queue.cc:1175] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d7e4cf694f71438298dd32a346abcf75" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20251028 02:59:51.757689 8553 raft_consensus.cc:2749] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.757895 8551 ts_tablet_manager.cc:1916] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.758983 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c5971fbd47e64ffda6df61dfc4756dfb
I20251028 02:59:51.759280 7967 catalog_manager.cc:6719] ReplaceTablet: tablet c5971fbd47e64ffda6df61dfc4756dfb deleted and replaced by tablet 6f38ce803e114831ace1412abdad7126
I20251028 02:59:51.760028 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet c5971fbd47e64ffda6df61dfc4756dfb with delete_type TABLET_DATA_DELETED (replaced by tablet 6f38ce803e114831ace1412abdad7126) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.760215 8551 ts_tablet_manager.cc:1929] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:51.760288 8551 log.cc:1199] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8643565320504fe6b6154e936e55fca9
I20251028 02:59:51.760542 8551 ts_tablet_manager.cc:1950] T 8643565320504fe6b6154e936e55fca9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.761147 8551 tablet_replica.cc:333] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.761269 8551 raft_consensus.cc:2243] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.761365 8551 raft_consensus.cc:2272] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.762679 8551 ts_tablet_manager.cc:1916] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.763882 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c6c40605a5a48d79837893443f4ca1b with delete_type TABLET_DATA_DELETED (replaced by tablet aa1fe079ad014fe0b0d99efe7981de88) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.764077 8552 tablet_replica.cc:333] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.764216 8552 raft_consensus.cc:2243] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.764336 8552 raft_consensus.cc:2272] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.764901 8552 ts_tablet_manager.cc:1916] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.765285 8551 ts_tablet_manager.cc:1929] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.765367 8551 log.cc:1199] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/6c6c40605a5a48d79837893443f4ca1b
I20251028 02:59:51.765568 8551 ts_tablet_manager.cc:1950] T 6c6c40605a5a48d79837893443f4ca1b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.767014 8552 ts_tablet_manager.cc:1929] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.767088 8552 log.cc:1199] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/6c6c40605a5a48d79837893443f4ca1b
I20251028 02:59:51.767290 8552 ts_tablet_manager.cc:1950] T 6c6c40605a5a48d79837893443f4ca1b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.767995 8553 raft_consensus.cc:493] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.768074 8553 raft_consensus.cc:515] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.768410 8553 leader_election.cc:290] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.768805 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9729143013a4ead9c3144747ecbe58c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:51.768941 8421 raft_consensus.cc:2393] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in this term.
I20251028 02:59:51.769276 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 909b08e0860b4e4f9bb1822acfd1dc02 with delete_type TABLET_DATA_DELETED (replaced by tablet 8b88fda3f40540f3b42ee7f33dd33251) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.770488 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9729143013a4ead9c3144747ecbe58c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.770973 8218 leader_election.cc:304] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.771091 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 8cb97d5192684d748b8b485f891b0dc0 (DEFAULT_TABLE table=default.Table_35fd4c0e400244f28d1a855779611c46 [id=a9662d8045574f0abf81c9e6bc2f180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.771238 8556 raft_consensus.cc:3060] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.771481 8261 tablet_service.cc:1505] Processing CreateTablet for tablet d248313db35e40e38e2c5e07e68a69af (DEFAULT_TABLE table=default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.771551 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 20102243a3514ad1b5dc520b714d2413 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.771714 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e5f0326a03040de81a506caa80dcbd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.771965 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 8cb97d5192684d748b8b485f891b0dc0 (DEFAULT_TABLE table=default.Table_35fd4c0e400244f28d1a855779611c46 [id=a9662d8045574f0abf81c9e6bc2f180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.772081 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e5f0326a03040de81a506caa80dcbd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.772127 8556 raft_consensus.cc:2749] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.772217 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cb97d5192684d748b8b485f891b0dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.772984 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet ed7d32bd6b554916af9327befda800e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 1c81698dcbe14d5ba61471e7249705db) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.771991 8398 tablet_service.cc:1505] Processing CreateTablet for tablet d248313db35e40e38e2c5e07e68a69af (DEFAULT_TABLE table=default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.773216 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d248313db35e40e38e2c5e07e68a69af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.774314 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cb97d5192684d748b8b485f891b0dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.774798 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 909b08e0860b4e4f9bb1822acfd1dc02 with delete_type TABLET_DATA_DELETED (replaced by tablet 8b88fda3f40540f3b42ee7f33dd33251) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.775070 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3df041f16a0448839351bbc768633b69
I20251028 02:59:51.775569 8782 tablet.cc:1722] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.775779 8146 raft_consensus.cc:1275] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:51.776213 8549 consensus_queue.cc:1048] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.776538 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d248313db35e40e38e2c5e07e68a69af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.776760 8753 tablet.cc:1722] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.775068 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 102fc67bcf7f4336a26220fdb4f7a4d9 (table default.Table_94d71cfa43134dff9a62343374aed8fe [id=7f26fae434ec4bdda077c8ceb5350aac]). Expected version 1 got 0
I20251028 02:59:51.778443 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 3df041f16a0448839351bbc768633b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:51.779351 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet c5971fbd47e64ffda6df61dfc4756dfb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5971fbd47e64ffda6df61dfc4756dfb already in progress: creating tablet
I20251028 02:59:51.779487 8550 tablet_replica.cc:333] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.779695 8550 raft_consensus.cc:2243] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.779809 8550 raft_consensus.cc:2272] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.780527 8550 ts_tablet_manager.cc:1916] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.780671 8546 tablet_bootstrap.cc:492] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.780769 8546 ts_tablet_manager.cc:1403] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.039s user 0.000s sys 0.002s
I20251028 02:59:51.781261 8546 raft_consensus.cc:359] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.781360 8546 raft_consensus.cc:385] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.781390 8546 raft_consensus.cc:740] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.781464 8616 raft_consensus.cc:493] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.781549 8616 raft_consensus.cc:515] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.781956 8616 leader_election.cc:290] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.781934 8726 tablet.cc:1722] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:51.782370 8261 tablet_service.cc:1505] Processing CreateTablet for tablet cfd68d91fd9b4c69871383e90bd2ff85 (DEFAULT_TABLE table=default.Table_620185a247324af6b2054263457ec7da [id=c33a0072d1384953b73c2e3d90176cc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.782487 8285 raft_consensus.cc:1275] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:51.782639 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd68d91fd9b4c69871383e90bd2ff85. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.782727 8546 consensus_queue.cc:260] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.783058 8546 ts_tablet_manager.cc:1434] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 02:59:51.783273 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ed7d32bd6b554916af9327befda800e2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.783553 8546 tablet_bootstrap.cc:492] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:51.784394 8354 leader_election.cc:343] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.784947 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 0868c69e28a840b4ab9dad61d50983ec (DEFAULT_TABLE table=default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.785179 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0868c69e28a840b4ab9dad61d50983ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.785614 8616 consensus_queue.cc:1048] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.787019 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 8b88fda3f40540f3b42ee7f33dd33251 (DEFAULT_TABLE table=default.Table_9c49ffa0346a4eaaabc68b637fcfc820 [id=98a07edd5f664674a2d33344038a8801]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.787273 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b88fda3f40540f3b42ee7f33dd33251. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.787348 8546 tablet_bootstrap.cc:654] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.788039 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet ab1ec08da9f64109a3fe9400fe4e366c with delete_type TABLET_DATA_DELETED (replaced by tablet 0e20b1f36d3f4f70a93ad8a0b29d6687) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.788338 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 9f1c1e14665c4712bffa439ab98b2a6e (DEFAULT_TABLE table=default.Table_ec3580a0e3af454f9154d6733e07ca55 [id=3eb0b70129be40e598874d492735ba51]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.788568 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f1c1e14665c4712bffa439ab98b2a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.789500 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 0868c69e28a840b4ab9dad61d50983ec (DEFAULT_TABLE table=default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.789731 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0868c69e28a840b4ab9dad61d50983ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.782119 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8643565320504fe6b6154e936e55fca9 (table default.Table_35fd4c0e400244f28d1a855779611c46 [id=a9662d8045574f0abf81c9e6bc2f180c]) successfully deleted
I20251028 02:59:51.790634 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 6c6c40605a5a48d79837893443f4ca1b (table default.Table_dff2d248a846455eb7bd3f3648f7ceee [id=77dd1eb6df0747bcab57b157ab8a850e]) successfully deleted
W20251028 02:59:51.790786 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 66fb0173e40e40999f90f015ddb294b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66fb0173e40e40999f90f015ddb294b7 already in progress: creating tablet
W20251028 02:59:51.790902 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f11c46ed489e41d3a25cfcd9413443af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f11c46ed489e41d3a25cfcd9413443af already in progress: creating tablet
I20251028 02:59:51.790937 8648 consensus_queue.cc:1048] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.791591 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 9f1c1e14665c4712bffa439ab98b2a6e (DEFAULT_TABLE table=default.Table_ec3580a0e3af454f9154d6733e07ca55 [id=3eb0b70129be40e598874d492735ba51]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.791728 8656 raft_consensus.cc:493] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.791817 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f1c1e14665c4712bffa439ab98b2a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.791832 8656 raft_consensus.cc:515] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.792208 8656 leader_election.cc:290] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.793916 7965 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_a28e8e2125f541a6944d94f73ee01cae [id=4eb2743632814baab2e39bbdce700d6b] alter complete (version 1)
I20251028 02:59:51.794289 8553 raft_consensus.cc:493] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.794378 8553 raft_consensus.cc:515] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.794726 8553 leader_election.cc:290] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:51.795171 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4132a2b416cc434fbaebfd7df7e5d878" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:51.795508 8220 leader_election.cc:343] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.795667 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f67b25b85194ce0be43f01adccd6c48" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:51.795771 8421 raft_consensus.cc:2468] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:51.795917 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet c5971fbd47e64ffda6df61dfc4756dfb with delete_type TABLET_DATA_DELETED (replaced by tablet 6f38ce803e114831ace1412abdad7126) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.796159 8083 leader_election.cc:304] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.796332 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 3df041f16a0448839351bbc768633b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.796406 8562 raft_consensus.cc:2804] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.796454 8562 raft_consensus.cc:493] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.796494 8562 raft_consensus.cc:3060] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.796916 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_af0da4b12fef412fa59f7ec5fa3fc660" } modify_external_catalogs: true
I20251028 02:59:51.797034 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_af0da4b12fef412fa59f7ec5fa3fc660" } modify_external_catalogs: true
I20251028 02:59:51.797590 8398 tablet_service.cc:1505] Processing CreateTablet for tablet cfd68d91fd9b4c69871383e90bd2ff85 (DEFAULT_TABLE table=default.Table_620185a247324af6b2054263457ec7da [id=c33a0072d1384953b73c2e3d90176cc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.797611 8562 raft_consensus.cc:515] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.797854 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfd68d91fd9b4c69871383e90bd2ff85. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.798053 8562 leader_election.cc:290] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.798295 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_828e8df9af6a48168a189935f63b2941" } modify_external_catalogs: true
I20251028 02:59:51.798388 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_828e8df9af6a48168a189935f63b2941" } modify_external_catalogs: true
I20251028 02:59:51.799139 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4132a2b416cc434fbaebfd7df7e5d878" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.799099 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_24016eb841cb46299d394b16f2d4f3b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.799484 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24016eb841cb46299d394b16f2d4f3b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.799611 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_97bcd2a2dcfa4b08af27c1db65c4e4e2" } new_table_name: "default.Table_54486b99eed547a4aff3e091082b0b2f" modify_external_catalogs: true
W20251028 02:59:51.799616 8218 leader_election.cc:343] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.799700 8218 leader_election.cc:304] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.800073 8553 raft_consensus.cc:2749] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.800493 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f67b25b85194ce0be43f01adccd6c48" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.800515 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_83adc71d84344ff686fc777f2f72d54b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.800592 8283 raft_consensus.cc:3060] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.800652 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_309f970a07c6425bb6e764dfb593bc80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.800832 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83adc71d84344ff686fc777f2f72d54b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:51.800866 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_309f970a07c6425bb6e764dfb593bc80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.801568 8283 raft_consensus.cc:2468] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:51.802013 8083 leader_election.cc:304] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:51.802181 8656 raft_consensus.cc:2804] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.802254 8656 raft_consensus.cc:697] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:51.802377 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f67b25b85194ce0be43f01adccd6c48" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:51.802382 8656 consensus_queue.cc:237] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.802479 8421 raft_consensus.cc:3060] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.802832 8550 ts_tablet_manager.cc:1929] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:51.802896 8550 log.cc:1199] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3df041f16a0448839351bbc768633b69
I20251028 02:59:51.803156 8550 ts_tablet_manager.cc:1950] T 3df041f16a0448839351bbc768633b69 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.803465 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2f67b25b85194ce0be43f01adccd6c48" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.803522 8421 raft_consensus.cc:2468] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:51.803525 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7ad84b67d9424145aa8f2efd29320e2f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.803586 8285 raft_consensus.cc:2376] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1. Re-sending same reply.
I20251028 02:59:51.803598 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet f11c46ed489e41d3a25cfcd9413443af (table default.Table_ec3580a0e3af454f9154d6733e07ca55 [id=3eb0b70129be40e598874d492735ba51]) successfully deleted
I20251028 02:59:51.803725 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 6c6c40605a5a48d79837893443f4ca1b (table default.Table_dff2d248a846455eb7bd3f3648f7ceee [id=77dd1eb6df0747bcab57b157ab8a850e]) successfully deleted
W20251028 02:59:51.803857 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ad84b67d9424145aa8f2efd29320e2f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:51.803886 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 3e5f0326a03040de81a506caa80dcbd9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3e5f0326a03040de81a506caa80dcbd9 already in progress: creating tablet
W20251028 02:59:51.804036 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 909b08e0860b4e4f9bb1822acfd1dc02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 909b08e0860b4e4f9bb1822acfd1dc02 already in progress: creating tablet
I20251028 02:59:51.804733 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 1ba114e5a94542a9b3662f28a93cd23a (DEFAULT_TABLE table=default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.805011 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ba114e5a94542a9b3662f28a93cd23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.804982 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 6f38ce803e114831ace1412abdad7126 (DEFAULT_TABLE table=default.Table_828e8df9af6a48168a189935f63b2941 [id=42c89dfccc714a9eb42e3779130c8af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.805298 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f38ce803e114831ace1412abdad7126. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.805325 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 3e7dbafe267a4c62b0418d1b26585aad (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:51.805907 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2fb1b38dc3684e8b9471ec10146a1a3d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.806188 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 1ba114e5a94542a9b3662f28a93cd23a (DEFAULT_TABLE table=default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:51.806210 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2fb1b38dc3684e8b9471ec10146a1a3d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.806416 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ba114e5a94542a9b3662f28a93cd23a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:51.807188 8356 leader_election.cc:343] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: ed7d32bd6b554916af9327befda800e2
I20251028 02:59:51.807278 8356 leader_election.cc:304] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.807998 8549 raft_consensus.cc:2749] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.808543 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 1ba114e5a94542a9b3662f28a93cd23a (DEFAULT_TABLE table=default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.808786 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ba114e5a94542a9b3662f28a93cd23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.809020 8552 tablet_replica.cc:333] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.809156 8552 raft_consensus.cc:2243] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.809254 8552 raft_consensus.cc:2272] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.809796 8552 ts_tablet_manager.cc:1916] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.810778 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]) successfully deleted
I20251028 02:59:51.811089 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet c5971fbd47e64ffda6df61dfc4756dfb with delete_type TABLET_DATA_DELETED (replaced by tablet 6f38ce803e114831ace1412abdad7126) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.811950 8552 ts_tablet_manager.cc:1929] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.812022 8552 log.cc:1199] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c5971fbd47e64ffda6df61dfc4756dfb
I20251028 02:59:51.812247 8552 ts_tablet_manager.cc:1950] T c5971fbd47e64ffda6df61dfc4756dfb P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.813720 8552 tablet_replica.cc:333] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.813941 8552 raft_consensus.cc:2243] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:51.814136 8552 raft_consensus.cc:2272] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.814208 8545 tablet_bootstrap.cc:492] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.814296 8545 ts_tablet_manager.cc:1403] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.061s user 0.001s sys 0.001s
I20251028 02:59:51.814335 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 6f38ce803e114831ace1412abdad7126 (DEFAULT_TABLE table=default.Table_828e8df9af6a48168a189935f63b2941 [id=42c89dfccc714a9eb42e3779130c8af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.814575 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f38ce803e114831ace1412abdad7126. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.814666 8552 ts_tablet_manager.cc:1916] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.814785 8545 raft_consensus.cc:359] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.814895 8545 raft_consensus.cc:385] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.814936 8545 raft_consensus.cc:740] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.815038 8545 consensus_queue.cc:260] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.816114 8545 ts_tablet_manager.cc:1434] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:51.816301 8545 tablet_bootstrap.cc:492] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.816982 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c5971fbd47e64ffda6df61dfc4756dfb (table default.Table_828e8df9af6a48168a189935f63b2941 [id=42c89dfccc714a9eb42e3779130c8af4]) successfully deleted
I20251028 02:59:51.817104 8552 ts_tablet_manager.cc:1929] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:51.817175 8552 log.cc:1199] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3df041f16a0448839351bbc768633b69
I20251028 02:59:51.817416 8552 ts_tablet_manager.cc:1950] T 3df041f16a0448839351bbc768633b69 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.817870 7964 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_94d71cfa43134dff9a62343374aed8fe [id=7f26fae434ec4bdda077c8ceb5350aac] alter complete (version 1)
I20251028 02:59:51.818856 8546 tablet_bootstrap.cc:492] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.818938 8546 ts_tablet_manager.cc:1403] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.002s
I20251028 02:59:51.818912 7964 catalog_manager.cc:5649] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:51.819608 8545 tablet_bootstrap.cc:654] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.819609 8546 raft_consensus.cc:359] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.819715 8546 raft_consensus.cc:385] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.819772 8546 raft_consensus.cc:740] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.819892 8546 consensus_queue.cc:260] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.820158 8546 ts_tablet_manager.cc:1434] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:51.820645 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 66fb0173e40e40999f90f015ddb294b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.820326 8546 tablet_bootstrap.cc:492] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.820844 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]) successfully deleted
I20251028 02:59:51.821036 8552 tablet_replica.cc:333] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:51.821185 8552 raft_consensus.cc:2243] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.821308 8552 raft_consensus.cc:2272] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.821856 8546 tablet_bootstrap.cc:654] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.822081 8549 consensus_queue.cc:1048] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.822588 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet fbc0337747a74e20b22aac654c2ab7cc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.822777 8552 ts_tablet_manager.cc:1916] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.824343 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 8b88fda3f40540f3b42ee7f33dd33251 (DEFAULT_TABLE table=default.Table_9c49ffa0346a4eaaabc68b637fcfc820 [id=98a07edd5f664674a2d33344038a8801]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.824605 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b88fda3f40540f3b42ee7f33dd33251. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.825387 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 3df041f16a0448839351bbc768633b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.825430 8552 ts_tablet_manager.cc:1929] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.825510 8552 log.cc:1199] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/66fb0173e40e40999f90f015ddb294b7
I20251028 02:59:51.825752 8552 ts_tablet_manager.cc:1950] T 66fb0173e40e40999f90f015ddb294b7 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.828146 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 3e7dbafe267a4c62b0418d1b26585aad (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:51.829304 8753 tablet.cc:1722] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:51.831323 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 6f38ce803e114831ace1412abdad7126 (DEFAULT_TABLE table=default.Table_828e8df9af6a48168a189935f63b2941 [id=42c89dfccc714a9eb42e3779130c8af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.831700 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f38ce803e114831ace1412abdad7126. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.834316 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a99123e464ae42ca9598a926c537455d
I20251028 02:59:51.835361 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6f38ce803e114831ace1412abdad7126
I20251028 02:59:51.835809 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1fec82ff069746a3b9b74be77d69357c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.836244 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fec82ff069746a3b9b74be77d69357c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:51.837128 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 consensus state has no leader
I20251028 02:59:51.838565 8616 raft_consensus.cc:493] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.838652 8616 raft_consensus.cc:515] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.839052 8616 leader_election.cc:290] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.839399 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "11a8056e97ff452e93e91622fedb77a6" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.839478 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet a99123e464ae42ca9598a926c537455d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.839640 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f38ce803e114831ace1412abdad7126 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.839725 8550 tablet_replica.cc:333] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.839871 8550 raft_consensus.cc:2243] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
W20251028 02:59:51.839893 8354 leader_election.cc:343] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.840036 8550 pending_rounds.cc:70] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:51.840014 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 66fb0173e40e40999f90f015ddb294b7 (table default.Table_eba1e050b6d54c418fbfec1ffcfc8052 [id=d2babb92c2514978b979b30ab6c511f8]) successfully deleted
I20251028 02:59:51.840083 8550 pending_rounds.cc:77] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597123837734912 op_type: NO_OP noop_request { }
I20251028 02:59:51.840147 8550 raft_consensus.cc:2889] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:51.840214 8550 raft_consensus.cc:2272] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.840389 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f38ce803e114831ace1412abdad7126 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.841111 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "11a8056e97ff452e93e91622fedb77a6" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.841239 8283 raft_consensus.cc:2468] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.841286 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f38ce803e114831ace1412abdad7126 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.841542 8356 leader_election.cc:304] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.841807 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet a99123e464ae42ca9598a926c537455d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:51.842003 8550 ts_tablet_manager.cc:1916] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.842463 8545 tablet_bootstrap.cc:492] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.842537 8545 ts_tablet_manager.cc:1403] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.000s
I20251028 02:59:51.843024 8545 raft_consensus.cc:359] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.843137 8545 raft_consensus.cc:385] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.843176 8545 raft_consensus.cc:740] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.843335 8545 consensus_queue.cc:260] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.843516 8648 raft_consensus.cc:2804] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.843575 8648 raft_consensus.cc:493] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.843617 8545 ts_tablet_manager.cc:1434] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.843636 8648 raft_consensus.cc:3060] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.843803 8545 tablet_bootstrap.cc:492] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.844395 8550 ts_tablet_manager.cc:1929] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:51.844466 8550 log.cc:1199] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a99123e464ae42ca9598a926c537455d
I20251028 02:59:51.844605 8648 raft_consensus.cc:515] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.844707 8550 ts_tablet_manager.cc:1950] T a99123e464ae42ca9598a926c537455d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.844882 8545 tablet_bootstrap.cc:654] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.844983 8648 leader_election.cc:290] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.845584 8551 tablet_replica.cc:333] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.845712 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a99123e464ae42ca9598a926c537455d (table default.Table_af0da4b12fef412fa59f7ec5fa3fc660 [id=d103d5f71ac24421bb60bf82e44dc5e9]) successfully deleted
I20251028 02:59:51.845715 8551 raft_consensus.cc:2243] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.846065 8551 raft_consensus.cc:2272] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.846531 8551 ts_tablet_manager.cc:1916] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:51.846807 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6f38ce803e114831ace1412abdad7126 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f38ce803e114831ace1412abdad7126 already in progress: creating tablet
I20251028 02:59:51.846931 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "11a8056e97ff452e93e91622fedb77a6" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:51.847240 8552 tablet_replica.cc:333] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 02:59:51.847325 8354 leader_election.cc:343] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.847412 8552 raft_consensus.cc:2243] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.847522 8552 raft_consensus.cc:2272] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.848367 8552 ts_tablet_manager.cc:1916] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.850229 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "11a8056e97ff452e93e91622fedb77a6" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.850344 8283 raft_consensus.cc:3060] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.850718 8552 ts_tablet_manager.cc:1929] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.850780 8552 log.cc:1199] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/a99123e464ae42ca9598a926c537455d
I20251028 02:59:51.850989 8552 ts_tablet_manager.cc:1950] T a99123e464ae42ca9598a926c537455d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:51.851349 8283 raft_consensus.cc:2468] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.851395 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet bd3cebaf966245b7b009b7dd90e2eb71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:51.851586 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 909b08e0860b4e4f9bb1822acfd1dc02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 909b08e0860b4e4f9bb1822acfd1dc02 already in progress: creating tablet
I20251028 02:59:51.851691 8356 leader_election.cc:304] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:51.851750 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 6f38ce803e114831ace1412abdad7126 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f38ce803e114831ace1412abdad7126 already in progress: creating tablet
I20251028 02:59:51.851859 8616 raft_consensus.cc:2804] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.851922 8616 raft_consensus.cc:697] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:51.851938 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet a99123e464ae42ca9598a926c537455d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.852046 8616 consensus_queue.cc:237] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.852427 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet a99123e464ae42ca9598a926c537455d (table default.Table_af0da4b12fef412fa59f7ec5fa3fc660 [id=d103d5f71ac24421bb60bf82e44dc5e9]) successfully deleted
I20251028 02:59:51.855621 8782 tablet.cc:1722] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:51.860920 8551 ts_tablet_manager.cc:1929] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.861023 8551 log.cc:1199] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/20102243a3514ad1b5dc520b714d2413
I20251028 02:59:51.861251 8551 ts_tablet_manager.cc:1950] T 20102243a3514ad1b5dc520b714d2413 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.861954 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 20102243a3514ad1b5dc520b714d2413 (table default.Table_50b58157f999483da285fc8730aba0e9 [id=3f05bbb6510e4149acbb9d0c236ee4f0]) successfully deleted
I20251028 02:59:51.863440 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 488bc52f1219478ab8843c9b1d79de2b (DEFAULT_TABLE table=default.Table_7ad84b67d9424145aa8f2efd29320e2f [id=d8299d574efc4f3f9f40b93d5db5e430]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.863513 8648 raft_consensus.cc:493] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.863607 8648 raft_consensus.cc:515] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.863710 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488bc52f1219478ab8843c9b1d79de2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.864018 8648 leader_election.cc:290] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.864759 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 83495009a1dc4f4388d2ae65f2dda548 (DEFAULT_TABLE table=default.Table_309f970a07c6425bb6e764dfb593bc80 [id=27a892441799474b9572251752e147f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.865024 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83495009a1dc4f4388d2ae65f2dda548. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.865738 8551 tablet_replica.cc:333] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.865741 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ab1ec08da9f64109a3fe9400fe4e366c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.865866 8283 raft_consensus.cc:2468] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.865895 8551 raft_consensus.cc:2243] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.866200 8356 leader_election.cc:304] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:51.866364 8616 raft_consensus.cc:493] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.866423 8616 raft_consensus.cc:515] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.866443 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ab1ec08da9f64109a3fe9400fe4e366c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.866725 8648 raft_consensus.cc:2804] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.866783 8648 raft_consensus.cc:493] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.866817 8648 raft_consensus.cc:3060] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.867048 8551 raft_consensus.cc:2272] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.867622 8551 ts_tablet_manager.cc:1916] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.867707 8648 raft_consensus.cc:515] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.868006 8648 leader_election.cc:290] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.868172 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4132a2b416cc434fbaebfd7df7e5d878" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.868283 8283 raft_consensus.cc:2468] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:51.868276 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 0703e18d7dba4d628e3cc310be55fcf5 (DEFAULT_TABLE table=default.Table_2fb1b38dc3684e8b9471ec10146a1a3d [id=d4eaebd005bc42dd86d7125ef555546f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.866739 8616 leader_election.cc:290] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.868507 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0703e18d7dba4d628e3cc310be55fcf5. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:51.868741 8354 leader_election.cc:343] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.869128 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4132a2b416cc434fbaebfd7df7e5d878" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.869128 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ab1ec08da9f64109a3fe9400fe4e366c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:51.869658 8354 leader_election.cc:343] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.869735 8545 tablet_bootstrap.cc:492] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
W20251028 02:59:51.869812 8354 leader_election.cc:343] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.869854 8545 ts_tablet_manager.cc:1403] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20251028 02:59:51.870007 8551 ts_tablet_manager.cc:1929] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.870080 8551 log.cc:1199] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ed7d32bd6b554916af9327befda800e2
I20251028 02:59:51.870312 8551 ts_tablet_manager.cc:1950] T ed7d32bd6b554916af9327befda800e2 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.870332 8545 raft_consensus.cc:359] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.870425 8545 raft_consensus.cc:385] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.870414 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ab1ec08da9f64109a3fe9400fe4e366c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.870458 8545 raft_consensus.cc:740] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.870522 8283 raft_consensus.cc:3060] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.870590 8545 consensus_queue.cc:260] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.870870 8545 ts_tablet_manager.cc:1434] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:51.870898 8551 tablet_replica.cc:333] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.871049 8545 tablet_bootstrap.cc:492] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.871486 8283 raft_consensus.cc:2468] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.872424 8545 tablet_bootstrap.cc:654] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:51.873322 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 3e5f0326a03040de81a506caa80dcbd9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3e5f0326a03040de81a506caa80dcbd9 already in progress: creating tablet
I20251028 02:59:51.873526 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 488bc52f1219478ab8843c9b1d79de2b (DEFAULT_TABLE table=default.Table_7ad84b67d9424145aa8f2efd29320e2f [id=d8299d574efc4f3f9f40b93d5db5e430]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.873526 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 83495009a1dc4f4388d2ae65f2dda548 (DEFAULT_TABLE table=default.Table_309f970a07c6425bb6e764dfb593bc80 [id=27a892441799474b9572251752e147f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.873813 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488bc52f1219478ab8843c9b1d79de2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.873962 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83495009a1dc4f4388d2ae65f2dda548. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.878381 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 0141ebd1ddad421bbb8eca7615e78351 (DEFAULT_TABLE table=default.Table_83adc71d84344ff686fc777f2f72d54b [id=0b22f43d0bda44b98e6f4b2f26153c5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.878659 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0141ebd1ddad421bbb8eca7615e78351. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.879010 8551 raft_consensus.cc:2243] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.879148 8551 raft_consensus.cc:2272] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.879817 8551 ts_tablet_manager.cc:1916] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.879798 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 0703e18d7dba4d628e3cc310be55fcf5 (DEFAULT_TABLE table=default.Table_2fb1b38dc3684e8b9471ec10146a1a3d [id=d4eaebd005bc42dd86d7125ef555546f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.880003 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 127e08303c564348b14257ec243418f4 (DEFAULT_TABLE table=default.Table_24016eb841cb46299d394b16f2d4f3b8 [id=72061aad76324293b5485166aac98a4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.880069 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0703e18d7dba4d628e3cc310be55fcf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.880185 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127e08303c564348b14257ec243418f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.880511 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 127e08303c564348b14257ec243418f4 (DEFAULT_TABLE table=default.Table_24016eb841cb46299d394b16f2d4f3b8 [id=72061aad76324293b5485166aac98a4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.880744 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127e08303c564348b14257ec243418f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.882186 8551 ts_tablet_manager.cc:1929] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.882170 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 0141ebd1ddad421bbb8eca7615e78351 (DEFAULT_TABLE table=default.Table_83adc71d84344ff686fc777f2f72d54b [id=0b22f43d0bda44b98e6f4b2f26153c5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.882251 8551 log.cc:1199] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ab1ec08da9f64109a3fe9400fe4e366c
I20251028 02:59:51.882369 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0141ebd1ddad421bbb8eca7615e78351. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.882452 8551 ts_tablet_manager.cc:1950] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.882882 8551 tablet_replica.cc:333] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.883010 8551 raft_consensus.cc:2243] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.883123 8551 raft_consensus.cc:2272] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.883626 8551 ts_tablet_manager.cc:1916] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.885571 8551 ts_tablet_manager.cc:1929] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.885648 8551 log.cc:1199] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c5971fbd47e64ffda6df61dfc4756dfb
I20251028 02:59:51.886440 8356 leader_election.cc:304] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.886631 8356 leader_election.cc:304] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:51.886693 8356 raft_consensus.cc:2686] T ab1ec08da9f64109a3fe9400fe4e366c P d7e4cf694f71438298dd32a346abcf75: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251028 02:59:51.886863 8546 tablet_bootstrap.cc:492] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.886957 8546 ts_tablet_manager.cc:1403] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.067s user 0.001s sys 0.001s
I20251028 02:59:51.887069 8726 tablet.cc:1722] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:51.887449 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 488bc52f1219478ab8843c9b1d79de2b (DEFAULT_TABLE table=default.Table_7ad84b67d9424145aa8f2efd29320e2f [id=d8299d574efc4f3f9f40b93d5db5e430]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.887610 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 0141ebd1ddad421bbb8eca7615e78351 (DEFAULT_TABLE table=default.Table_83adc71d84344ff686fc777f2f72d54b [id=0b22f43d0bda44b98e6f4b2f26153c5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.887682 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488bc52f1219478ab8843c9b1d79de2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.887786 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0141ebd1ddad421bbb8eca7615e78351. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.889180 8546 raft_consensus.cc:359] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.889298 8546 raft_consensus.cc:385] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.889331 8546 raft_consensus.cc:740] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.889462 8546 consensus_queue.cc:260] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.889714 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 83495009a1dc4f4388d2ae65f2dda548 (DEFAULT_TABLE table=default.Table_309f970a07c6425bb6e764dfb593bc80 [id=27a892441799474b9572251752e147f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.889801 8546 ts_tablet_manager.cc:1434] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:51.889881 8648 raft_consensus.cc:2804] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.889927 8648 raft_consensus.cc:493] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.889967 8648 raft_consensus.cc:3060] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.890002 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83495009a1dc4f4388d2ae65f2dda548. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.890013 8546 tablet_bootstrap.cc:492] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.890779 8648 raft_consensus.cc:515] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.891057 8546 tablet_bootstrap.cc:654] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.891165 8648 leader_election.cc:290] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.892006 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4132a2b416cc434fbaebfd7df7e5d878" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:51.892038 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 0703e18d7dba4d628e3cc310be55fcf5 (DEFAULT_TABLE table=default.Table_2fb1b38dc3684e8b9471ec10146a1a3d [id=d4eaebd005bc42dd86d7125ef555546f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.892274 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0703e18d7dba4d628e3cc310be55fcf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.892663 7964 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a] alter complete (version 2)
W20251028 02:59:51.892999 8354 leader_election.cc:343] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.894099 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ed7d32bd6b554916af9327befda800e2 (table default.Table_310089f9c2b34a7e9e1c01c97fdb9b11 [id=c103fff3458d4d08affefb7c38d35e52]) successfully deleted
I20251028 02:59:51.894236 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ab1ec08da9f64109a3fe9400fe4e366c (table default.Table_54486b99eed547a4aff3e091082b0b2f [id=7a2a6ab3c9744942a9413c6cb057f527]) successfully deleted
I20251028 02:59:51.894299 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 127e08303c564348b14257ec243418f4 (DEFAULT_TABLE table=default.Table_24016eb841cb46299d394b16f2d4f3b8 [id=72061aad76324293b5485166aac98a4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:51.894531 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127e08303c564348b14257ec243418f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:51.895352 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4132a2b416cc434fbaebfd7df7e5d878" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:51.895473 8283 raft_consensus.cc:3060] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.897864 8283 raft_consensus.cc:2468] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:51.899109 8356 leader_election.cc:304] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.899371 8648 raft_consensus.cc:2804] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.899456 8648 raft_consensus.cc:697] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:51.899588 8648 consensus_queue.cc:237] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.900708 8551 ts_tablet_manager.cc:1950] T c5971fbd47e64ffda6df61dfc4756dfb P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.901350 8551 tablet_replica.cc:333] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.901503 8551 raft_consensus.cc:2243] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.901614 8551 raft_consensus.cc:2272] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.902192 8551 ts_tablet_manager.cc:1916] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.902397 8546 tablet_bootstrap.cc:492] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.902518 8546 ts_tablet_manager.cc:1403] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20251028 02:59:51.903141 8546 raft_consensus.cc:359] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.903213 8544 tablet_bootstrap.cc:492] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.903259 8546 raft_consensus.cc:385] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.903326 8544 ts_tablet_manager.cc:1403] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.155s user 0.002s sys 0.001s
I20251028 02:59:51.903189 7964 catalog_manager.cc:5649] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:51.903421 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 11a8056e97ff452e93e91622fedb77a6 (table default.Table_fe2e8997c049473bb57e49992eeec9fa [id=7fe387bad5e740429e327d42f1efde22]). Expected version 3 got 0
I20251028 02:59:51.903543 8546 raft_consensus.cc:740] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.903695 8546 consensus_queue.cc:260] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.903810 8544 raft_consensus.cc:359] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.903940 8544 raft_consensus.cc:385] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.903987 8544 raft_consensus.cc:740] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.904007 8546 ts_tablet_manager.cc:1434] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:51.904134 8544 consensus_queue.cc:260] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.904207 8546 tablet_bootstrap.cc:492] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.904413 8544 ts_tablet_manager.cc:1434] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.904592 8544 tablet_bootstrap.cc:492] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.905261 8546 tablet_bootstrap.cc:654] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.905591 8544 tablet_bootstrap.cc:654] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.909108 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 9e908d5921d34c9396c1ccdc857c66b7 (table default.Table_a28e8e2125f541a6944d94f73ee01cae [id=4eb2743632814baab2e39bbdce700d6b]). Expected version 1 got 0
I20251028 02:59:51.910259 8551 ts_tablet_manager.cc:1929] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.910341 8551 log.cc:1199] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:51.910552 8551 ts_tablet_manager.cc:1950] T fbc0337747a74e20b22aac654c2ab7cc P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.911026 8545 tablet_bootstrap.cc:492] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.911116 8545 ts_tablet_manager.cc:1403] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.000s
I20251028 02:59:51.911232 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet fbc0337747a74e20b22aac654c2ab7cc (table default.Table_1cdf4b1794504307906ac65e49eb2c71 [id=b34ab58e43e943909c9bdd79ad69e639]) successfully deleted
I20251028 02:59:51.911643 8551 tablet_replica.cc:333] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.911710 8545 raft_consensus.cc:359] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.911772 8551 raft_consensus.cc:2243] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.911803 8545 raft_consensus.cc:385] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.911834 8545 raft_consensus.cc:740] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.911878 8551 raft_consensus.cc:2272] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.911947 8545 consensus_queue.cc:260] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.912240 8545 ts_tablet_manager.cc:1434] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.912428 8545 tablet_bootstrap.cc:492] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.912549 8551 ts_tablet_manager.cc:1916] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.912886 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 9e908d5921d34c9396c1ccdc857c66b7 (table default.Table_a28e8e2125f541a6944d94f73ee01cae [id=4eb2743632814baab2e39bbdce700d6b]). Expected version 1 got 0
I20251028 02:59:51.914371 8546 tablet_bootstrap.cc:492] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.914484 8546 ts_tablet_manager.cc:1403] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:51.915009 8546 raft_consensus.cc:359] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.915136 8546 raft_consensus.cc:385] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.915174 8546 raft_consensus.cc:740] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.916392 8551 ts_tablet_manager.cc:1929] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:51.916465 8551 log.cc:1199] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3df041f16a0448839351bbc768633b69
I20251028 02:59:51.916710 8551 ts_tablet_manager.cc:1950] T 3df041f16a0448839351bbc768633b69 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.917276 8546 consensus_queue.cc:260] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.917654 8546 ts_tablet_manager.cc:1434] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:51.917771 7961 catalog_manager.cc:5649] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:51.918449 8283 raft_consensus.cc:1275] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:51.918789 8648 consensus_queue.cc:1048] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:51.919381 8354 consensus_peers.cc:597] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 -> Peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Couldn't send request to peer fbd2afb4894c4e7a8520cc09f8593aa0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:51.921123 8562 raft_consensus.cc:493] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.921228 8562 raft_consensus.cc:515] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.921677 8562 leader_election.cc:290] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:51.922268 8546 tablet_bootstrap.cc:492] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.922248 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e908d5921d34c9396c1ccdc857c66b7" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:51.922464 8545 tablet_bootstrap.cc:654] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.922626 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3df041f16a0448839351bbc768633b69 (table default.Table_78628adeb83d46c1913ffa57d5336dcc [id=a277ab5f37b54d53816ab230cf0690a3]) successfully deleted
I20251028 02:59:51.922869 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e908d5921d34c9396c1ccdc857c66b7" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:51.923360 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 6f38ce803e114831ace1412abdad7126 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f38ce803e114831ace1412abdad7126 already in progress: creating tablet
I20251028 02:59:51.923578 8551 tablet_replica.cc:333] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.923722 8551 raft_consensus.cc:2243] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.923838 8551 raft_consensus.cc:2272] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.923959 8083 leader_election.cc:304] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:51.924232 8562 raft_consensus.cc:2749] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:51.924495 8551 ts_tablet_manager.cc:1916] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.925714 8544 tablet_bootstrap.cc:492] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.925822 8544 ts_tablet_manager.cc:1403] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20251028 02:59:51.925875 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet f4bebc4575e4462cbc8a02fca4fb228b (table default.Table_69a042aded404f529721685d60c4f8b0 [id=e7c4b52cbf8c496e94ca8634d53e41de]). Expected version 2 got 0
I20251028 02:59:51.926345 8544 raft_consensus.cc:359] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.926447 8544 raft_consensus.cc:385] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.926476 8544 raft_consensus.cc:740] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.926591 8544 consensus_queue.cc:260] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.926880 8544 ts_tablet_manager.cc:1434] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.926934 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_69a042aded404f529721685d60c4f8b0" } new_table_name: "default.Table_deda042aff7640a3afd2f724eba68912" modify_external_catalogs: true
I20251028 02:59:51.927112 8544 tablet_bootstrap.cc:492] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.928171 8544 tablet_bootstrap.cc:654] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.930898 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1fec82ff069746a3b9b74be77d69357c" } new_table_name: "default.Table_3d8776d793a54bd795fb38bd45442f51" modify_external_catalogs: true
I20251028 02:59:51.931701 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet fbc0337747a74e20b22aac654c2ab7cc from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.932325 8551 ts_tablet_manager.cc:1929] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.932420 8551 log.cc:1199] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/bd3cebaf966245b7b009b7dd90e2eb71
I20251028 02:59:51.932621 8551 ts_tablet_manager.cc:1950] T bd3cebaf966245b7b009b7dd90e2eb71 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.933247 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet bd3cebaf966245b7b009b7dd90e2eb71 (table default.Table_6532259aa4cc47ee85ab998f23912c02 [id=960dc85e288d4ae6936f98938e123cf9]) successfully deleted
I20251028 02:59:51.933836 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet ed7d32bd6b554916af9327befda800e2 with delete_type TABLET_DATA_DELETED (replaced by tablet 1c81698dcbe14d5ba61471e7249705db) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.933954 8545 tablet_bootstrap.cc:492] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.934037 8545 ts_tablet_manager.cc:1403] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20251028 02:59:51.934196 8544 tablet_bootstrap.cc:492] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.934278 8544 ts_tablet_manager.cc:1403] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:51.934511 8545 raft_consensus.cc:359] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.934612 8550 tablet_replica.cc:333] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.934612 8545 raft_consensus.cc:385] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.934649 8545 raft_consensus.cc:740] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.934741 8550 raft_consensus.cc:2243] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.934767 8545 consensus_queue.cc:260] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.934831 8550 raft_consensus.cc:2272] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.934768 8544 raft_consensus.cc:359] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.934898 8544 raft_consensus.cc:385] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.934937 8544 raft_consensus.cc:740] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.935050 8545 ts_tablet_manager.cc:1434] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.935065 8544 consensus_queue.cc:260] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.935247 8545 tablet_bootstrap.cc:492] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.935360 8544 ts_tablet_manager.cc:1434] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.936312 8545 tablet_bootstrap.cc:654] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:51.937794 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f4bebc4575e4462cbc8a02fca4fb228b consensus state has no leader
I20251028 02:59:51.938321 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet f11c46ed489e41d3a25cfcd9413443af with delete_type TABLET_DATA_DELETED (replaced by tablet 9f1c1e14665c4712bffa439ab98b2a6e) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.938516 8551 tablet_replica.cc:333] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.938648 8551 raft_consensus.cc:2243] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.938760 8551 raft_consensus.cc:2272] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.939239 8551 ts_tablet_manager.cc:1916] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.940639 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7218cf040e3a418fa8c1ba0db0d4c1fe" } new_table_name: "default.Table_97b3a76d004f47c981178cf580a95985" modify_external_catalogs: true
I20251028 02:59:51.940762 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2ca85f32e093464c8bfade073ee1e7c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.941169 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ca85f32e093464c8bfade073ee1e7c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.942124 8544 tablet_bootstrap.cc:492] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.942198 8551 ts_tablet_manager.cc:1929] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.942274 8551 log.cc:1199] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a99123e464ae42ca9598a926c537455d
I20251028 02:59:51.942476 8551 ts_tablet_manager.cc:1950] T a99123e464ae42ca9598a926c537455d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.942571 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7fb744c8c91e4c9daadacdab1a408fa1" } modify_external_catalogs: true
I20251028 02:59:51.942679 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7fb744c8c91e4c9daadacdab1a408fa1" } modify_external_catalogs: true
I20251028 02:59:51.943548 8544 tablet_bootstrap.cc:654] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.944639 8551 tablet_replica.cc:333] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.944706 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a99123e464ae42ca9598a926c537455d (table default.Table_af0da4b12fef412fa59f7ec5fa3fc660 [id=d103d5f71ac24421bb60bf82e44dc5e9]) successfully deleted
I20251028 02:59:51.944766 8551 raft_consensus.cc:2243] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.944859 8551 raft_consensus.cc:2272] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.945577 8545 tablet_bootstrap.cc:492] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.945669 8545 ts_tablet_manager.cc:1403] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:51.946059 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_013e772cd60943509e2ea1c830dfe151"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.946213 8545 raft_consensus.cc:359] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.946328 8545 raft_consensus.cc:385] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.946399 8545 raft_consensus.cc:740] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
W20251028 02:59:51.946473 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_013e772cd60943509e2ea1c830dfe151 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.946522 8545 consensus_queue.cc:260] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.946833 8545 ts_tablet_manager.cc:1434] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.947050 8545 tablet_bootstrap.cc:492] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.947690 8546 tablet_bootstrap.cc:654] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.948021 8545 tablet_bootstrap.cc:654] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.946313 8551 ts_tablet_manager.cc:1916] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.948881 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 3209160f7b0e4f1a8c8121725454d088 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.948941 8550 ts_tablet_manager.cc:1916] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.949862 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 66fb0173e40e40999f90f015ddb294b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:51.950984 8550 ts_tablet_manager.cc:1929] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.951046 8550 log.cc:1199] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ed7d32bd6b554916af9327befda800e2
I20251028 02:59:51.951244 8550 ts_tablet_manager.cc:1950] T ed7d32bd6b554916af9327befda800e2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.952078 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet ab1ec08da9f64109a3fe9400fe4e366c with delete_type TABLET_DATA_DELETED (replaced by tablet 0e20b1f36d3f4f70a93ad8a0b29d6687) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.952549 8550 tablet_replica.cc:333] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.952662 8550 raft_consensus.cc:2243] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.952685 8546 tablet_bootstrap.cc:492] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.952742 8550 raft_consensus.cc:2272] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.952787 8546 ts_tablet_manager.cc:1403] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20251028 02:59:51.952996 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet ed7d32bd6b554916af9327befda800e2 (table default.Table_310089f9c2b34a7e9e1c01c97fdb9b11 [id=c103fff3458d4d08affefb7c38d35e52]) successfully deleted
I20251028 02:59:51.953308 8546 raft_consensus.cc:359] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.953429 8546 raft_consensus.cc:385] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.953472 8546 raft_consensus.cc:740] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.953619 8546 consensus_queue.cc:260] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.953904 8546 ts_tablet_manager.cc:1434] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.954142 8546 tablet_bootstrap.cc:492] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.954557 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2cd00f3aa8f744bd890e7fc0efb34753"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:51.955020 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2cd00f3aa8f744bd890e7fc0efb34753 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:51.955180 8546 tablet_bootstrap.cc:654] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.957134 8550 ts_tablet_manager.cc:1916] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.957695 8551 ts_tablet_manager.cc:1929] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.957777 8551 log.cc:1199] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/f11c46ed489e41d3a25cfcd9413443af
I20251028 02:59:51.958002 8551 ts_tablet_manager.cc:1950] T f11c46ed489e41d3a25cfcd9413443af P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.959018 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet f11c46ed489e41d3a25cfcd9413443af (table default.Table_ec3580a0e3af454f9154d6733e07ca55 [id=3eb0b70129be40e598874d492735ba51]) successfully deleted
I20251028 02:59:51.959167 8545 tablet_bootstrap.cc:492] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.959246 8545 ts_tablet_manager.cc:1403] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:51.959744 8550 ts_tablet_manager.cc:1929] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.959825 8550 log.cc:1199] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ab1ec08da9f64109a3fe9400fe4e366c
I20251028 02:59:51.959764 8545 raft_consensus.cc:359] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.959877 8545 raft_consensus.cc:385] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.959929 8545 raft_consensus.cc:740] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.960063 8550 ts_tablet_manager.cc:1950] T ab1ec08da9f64109a3fe9400fe4e366c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:51.960052 8545 consensus_queue.cc:260] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.960345 8545 ts_tablet_manager.cc:1434] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.960471 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet ab1ec08da9f64109a3fe9400fe4e366c (table default.Table_54486b99eed547a4aff3e091082b0b2f [id=7a2a6ab3c9744942a9413c6cb057f527]) successfully deleted
I20251028 02:59:51.960541 8545 tablet_bootstrap.cc:492] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.961505 8545 tablet_bootstrap.cc:654] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.962229 8549 consensus_queue.cc:1048] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.964661 8553 raft_consensus.cc:493] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.964761 8553 raft_consensus.cc:515] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.964968 8545 tablet_bootstrap.cc:492] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:51.965063 8545 ts_tablet_manager.cc:1403] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:51.965143 8553 leader_election.cc:290] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:51.965507 8545 raft_consensus.cc:359] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.965612 8545 raft_consensus.cc:385] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.965646 8545 raft_consensus.cc:740] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.965794 8545 consensus_queue.cc:260] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.966171 8545 ts_tablet_manager.cc:1434] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.966359 8545 tablet_bootstrap.cc:492] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:51.966428 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71909cad34b044b093a94397c4b9f29c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:51.966540 8421 raft_consensus.cc:2468] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:51.967536 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71909cad34b044b093a94397c4b9f29c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:51.967585 8220 leader_election.cc:304] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:51.967758 8553 raft_consensus.cc:2804] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:51.967828 8553 raft_consensus.cc:493] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:51.967878 8553 raft_consensus.cc:3060] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:51.967928 8218 leader_election.cc:343] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.968132 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 32ed23fd3cdc4be9ab0e65633c86e787 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.968421 8545 tablet_bootstrap.cc:654] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.968941 8553 raft_consensus.cc:515] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.969307 8553 leader_election.cc:290] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:51.969937 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71909cad34b044b093a94397c4b9f29c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:51.970366 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71909cad34b044b093a94397c4b9f29c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:51.970476 8421 raft_consensus.cc:3060] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:51.970770 8546 tablet_bootstrap.cc:492] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.970865 8546 ts_tablet_manager.cc:1403] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20251028 02:59:51.971035 8544 tablet_bootstrap.cc:492] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.971123 8544 ts_tablet_manager.cc:1403] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 02:59:51.971377 8546 raft_consensus.cc:359] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.971495 8546 raft_consensus.cc:385] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.971478 8421 raft_consensus.cc:2468] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:51.971529 8546 raft_consensus.cc:740] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.971585 8544 raft_consensus.cc:359] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.971693 8544 raft_consensus.cc:385] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.971657 8546 consensus_queue.cc:260] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.971726 8544 raft_consensus.cc:740] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.971836 8220 leader_election.cc:304] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:51.971846 8544 consensus_queue.cc:260] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.971984 8546 ts_tablet_manager.cc:1434] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:51.972062 8556 raft_consensus.cc:2804] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:51.972110 8544 ts_tablet_manager.cc:1434] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.972122 8556 raft_consensus.cc:697] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:51.972163 8546 tablet_bootstrap.cc:492] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.972229 8556 consensus_queue.cc:237] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.972309 8544 tablet_bootstrap.cc:492] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.973228 8546 tablet_bootstrap.cc:654] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.973261 8544 tablet_bootstrap.cc:654] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.973997 8551 tablet_replica.cc:333] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:51.974202 8551 raft_consensus.cc:2243] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.974305 8551 raft_consensus.cc:2272] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:51.974349 8218 leader_election.cc:343] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.975096 8551 ts_tablet_manager.cc:1916] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.975433 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet a770dec97abb478b8c5504cc007b6092 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:51.977063 8546 tablet_bootstrap.cc:492] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:51.977151 8546 ts_tablet_manager.cc:1403] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20251028 02:59:51.977743 8546 raft_consensus.cc:359] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.977859 8546 raft_consensus.cc:385] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.977906 8546 raft_consensus.cc:740] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:51.978020 8546 consensus_queue.cc:260] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.978295 8546 ts_tablet_manager.cc:1434] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:51.978472 8546 tablet_bootstrap.cc:492] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:51.979533 8648 consensus_queue.cc:1048] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.980211 8546 tablet_bootstrap.cc:654] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.982690 8544 tablet_bootstrap.cc:492] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:51.982797 8544 ts_tablet_manager.cc:1403] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:51.984901 8544 raft_consensus.cc:359] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.985033 8544 raft_consensus.cc:385] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:51.985071 8544 raft_consensus.cc:740] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
W20251028 02:59:51.985221 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 025a4d3e71ff443c8bdf649055a17d54 consensus state has no leader
I20251028 02:59:51.985200 8544 consensus_queue.cc:260] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:51.985514 8544 ts_tablet_manager.cc:1434] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:51.985710 8544 tablet_bootstrap.cc:492] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:51.986536 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet fbc0337747a74e20b22aac654c2ab7cc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.986702 8550 tablet_replica.cc:333] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.986830 8550 raft_consensus.cc:2243] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.987026 8550 raft_consensus.cc:2272] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.987030 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_620185a247324af6b2054263457ec7da" } modify_external_catalogs: true
I20251028 02:59:51.987149 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_620185a247324af6b2054263457ec7da" } modify_external_catalogs: true
I20251028 02:59:51.986810 8544 tablet_bootstrap.cc:654] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:51.989490 8550 ts_tablet_manager.cc:1916] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:51.991521 8551 ts_tablet_manager.cc:1929] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.991624 8551 log.cc:1199] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/66fb0173e40e40999f90f015ddb294b7
I20251028 02:59:51.991647 8550 ts_tablet_manager.cc:1929] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:51.991710 8550 log.cc:1199] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:51.991863 8551 ts_tablet_manager.cc:1950] T 66fb0173e40e40999f90f015ddb294b7 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:51.991916 8550 ts_tablet_manager.cc:1950] T fbc0337747a74e20b22aac654c2ab7cc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 02:59:51.992039 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99968de685b44cbfa4c002a6c40e9cb3 consensus state has no leader
I20251028 02:59:51.992565 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet fbc0337747a74e20b22aac654c2ab7cc (table default.Table_1cdf4b1794504307906ac65e49eb2c71 [id=b34ab58e43e943909c9bdd79ad69e639]) successfully deleted
I20251028 02:59:51.993125 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 66fb0173e40e40999f90f015ddb294b7 (table default.Table_eba1e050b6d54c418fbfec1ffcfc8052 [id=d2babb92c2514978b979b30ab6c511f8]) successfully deleted
I20251028 02:59:51.993472 8562 consensus_queue.cc:1048] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:51.994513 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet bd3cebaf966245b7b009b7dd90e2eb71 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:51.994675 8550 tablet_replica.cc:333] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:51.994801 8550 raft_consensus.cc:2243] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:51.994925 8550 raft_consensus.cc:2272] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:51.995329 8553 raft_consensus.cc:493] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:51.995410 8553 raft_consensus.cc:515] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:51.995747 8553 leader_election.cc:290] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:51.995757 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_053b1ca3666d4641b101c8c10f846980"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:51.996181 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc61d4b98fa34f8c9aa0656aa43bc25c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:51.996277 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_053b1ca3666d4641b101c8c10f846980 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:51.996515 8218 leader_election.cc:343] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.996980 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc61d4b98fa34f8c9aa0656aa43bc25c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:51.997344 8220 leader_election.cc:343] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:51.997421 8220 leader_election.cc:304] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:51.998942 8553 raft_consensus.cc:2749] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:52.002866 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_62299eb76b984e9ebf1136d16e827135" } new_table_name: "default.Table_5233b8b0ea95426a8497f5cbd89716dc" modify_external_catalogs: true
I20251028 02:59:52.005041 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4c1aacffea564931bb49e6bc29c6b0b7" } modify_external_catalogs: true
I20251028 02:59:52.005113 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3209160f7b0e4f1a8c8121725454d088
I20251028 02:59:52.005167 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4c1aacffea564931bb49e6bc29c6b0b7" } modify_external_catalogs: true
I20251028 02:59:52.005362 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 3209160f7b0e4f1a8c8121725454d088 deleted and replaced by tablet f57b4fa3ceec4557bc6d6a2125d80790
I20251028 02:59:52.005472 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 482d7541b350426f9ff16448a24daeed from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.006187 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 32ed23fd3cdc4be9ab0e65633c86e787
I20251028 02:59:52.006461 7966 catalog_manager.cc:6719] ReplaceTablet: tablet 32ed23fd3cdc4be9ab0e65633c86e787 deleted and replaced by tablet 2be7be7c732e4780859cbe6624edf54e
I20251028 02:59:52.006475 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 3209160f7b0e4f1a8c8121725454d088 with delete_type TABLET_DATA_DELETED (replaced by tablet f57b4fa3ceec4557bc6d6a2125d80790) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.006644 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 3209160f7b0e4f1a8c8121725454d088 with delete_type TABLET_DATA_DELETED (replaced by tablet f57b4fa3ceec4557bc6d6a2125d80790) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.006892 8552 tablet_replica.cc:333] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.006866 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 3209160f7b0e4f1a8c8121725454d088 with delete_type TABLET_DATA_DELETED (replaced by tablet f57b4fa3ceec4557bc6d6a2125d80790) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.007033 8552 raft_consensus.cc:2243] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.007092 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8b88fda3f40540f3b42ee7f33dd33251
I20251028 02:59:52.007166 8552 raft_consensus.cc:2272] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.007761 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b88fda3f40540f3b42ee7f33dd33251 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.007951 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 32ed23fd3cdc4be9ab0e65633c86e787 with delete_type TABLET_DATA_DELETED (replaced by tablet 2be7be7c732e4780859cbe6624edf54e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.008158 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 32ed23fd3cdc4be9ab0e65633c86e787 with delete_type TABLET_DATA_DELETED (replaced by tablet 2be7be7c732e4780859cbe6624edf54e) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.008205 8551 tablet_replica.cc:333] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.008261 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b88fda3f40540f3b42ee7f33dd33251 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.008342 8551 raft_consensus.cc:2243] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:52.008417 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b88fda3f40540f3b42ee7f33dd33251 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.008562 8551 raft_consensus.cc:2272] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.008920 8552 ts_tablet_manager.cc:1916] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.009125 8551 ts_tablet_manager.cc:1916] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.009316 8656 consensus_queue.cc:1048] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.011586 8551 ts_tablet_manager.cc:1929] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.011672 8551 log.cc:1199] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3209160f7b0e4f1a8c8121725454d088
I20251028 02:59:52.011981 8552 ts_tablet_manager.cc:1929] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.012058 8552 log.cc:1199] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3209160f7b0e4f1a8c8121725454d088
I20251028 02:59:52.012300 8552 ts_tablet_manager.cc:1950] T 3209160f7b0e4f1a8c8121725454d088 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.012809 8551 ts_tablet_manager.cc:1950] T 3209160f7b0e4f1a8c8121725454d088 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.013023 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3209160f7b0e4f1a8c8121725454d088 (table default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]) successfully deleted
I20251028 02:59:52.013559 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3209160f7b0e4f1a8c8121725454d088 (table default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]) successfully deleted
W20251028 02:59:52.013746 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 8b88fda3f40540f3b42ee7f33dd33251 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b88fda3f40540f3b42ee7f33dd33251 already in progress: creating tablet
I20251028 02:59:52.014055 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 32ed23fd3cdc4be9ab0e65633c86e787 with delete_type TABLET_DATA_DELETED (replaced by tablet 2be7be7c732e4780859cbe6624edf54e) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:52.014184 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 32ed23fd3cdc4be9ab0e65633c86e787 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32ed23fd3cdc4be9ab0e65633c86e787 already in progress: creating tablet
I20251028 02:59:52.014899 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 1c81698dcbe14d5ba61471e7249705db (table default.Table_310089f9c2b34a7e9e1c01c97fdb9b11 [id=c103fff3458d4d08affefb7c38d35e52]). Expected version 1 got 0
I20251028 02:59:52.015012 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 11a8056e97ff452e93e91622fedb77a6 (table default.Table_fe2e8997c049473bb57e49992eeec9fa [id=7fe387bad5e740429e327d42f1efde22]). Expected version 3 got 0
I20251028 02:59:52.017400 8550 ts_tablet_manager.cc:1916] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.019663 8550 ts_tablet_manager.cc:1929] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.019742 8550 log.cc:1199] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/bd3cebaf966245b7b009b7dd90e2eb71
W20251028 02:59:52.019812 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 32ed23fd3cdc4be9ab0e65633c86e787 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32ed23fd3cdc4be9ab0e65633c86e787 already in progress: creating tablet
I20251028 02:59:52.019965 8550 ts_tablet_manager.cc:1950] T bd3cebaf966245b7b009b7dd90e2eb71 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.020566 8550 tablet_replica.cc:333] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.020685 8550 raft_consensus.cc:2243] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.020784 8550 raft_consensus.cc:2272] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.021180 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a770dec97abb478b8c5504cc007b6092
I20251028 02:59:52.021245 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet bd3cebaf966245b7b009b7dd90e2eb71 (table default.Table_6532259aa4cc47ee85ab998f23912c02 [id=960dc85e288d4ae6936f98938e123cf9]) successfully deleted
I20251028 02:59:52.021463 7962 catalog_manager.cc:6719] ReplaceTablet: tablet a770dec97abb478b8c5504cc007b6092 deleted and replaced by tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906
I20251028 02:59:52.021772 8550 ts_tablet_manager.cc:1916] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.021836 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet a770dec97abb478b8c5504cc007b6092 with delete_type TABLET_DATA_DELETED (replaced by tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.022034 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet a770dec97abb478b8c5504cc007b6092 with delete_type TABLET_DATA_DELETED (replaced by tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:52.022548 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a770dec97abb478b8c5504cc007b6092 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a770dec97abb478b8c5504cc007b6092 already in progress: creating tablet
I20251028 02:59:52.023494 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet a770dec97abb478b8c5504cc007b6092 with delete_type TABLET_DATA_DELETED (replaced by tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:52.023571 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 8b88fda3f40540f3b42ee7f33dd33251 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b88fda3f40540f3b42ee7f33dd33251 already in progress: creating tablet
W20251028 02:59:52.023728 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a770dec97abb478b8c5504cc007b6092 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a770dec97abb478b8c5504cc007b6092 already in progress: creating tablet
I20251028 02:59:52.024492 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 99968de685b44cbfa4c002a6c40e9cb3 (DEFAULT_TABLE table=default.Table_3d8776d793a54bd795fb38bd45442f51 [id=be1f381c3ca048d891f5d175fa72fc65]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.024586 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 99968de685b44cbfa4c002a6c40e9cb3 (DEFAULT_TABLE table=default.Table_3d8776d793a54bd795fb38bd45442f51 [id=be1f381c3ca048d891f5d175fa72fc65]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.024693 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 99968de685b44cbfa4c002a6c40e9cb3 (DEFAULT_TABLE table=default.Table_3d8776d793a54bd795fb38bd45442f51 [id=be1f381c3ca048d891f5d175fa72fc65]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.024791 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99968de685b44cbfa4c002a6c40e9cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.024848 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99968de685b44cbfa4c002a6c40e9cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.024917 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99968de685b44cbfa4c002a6c40e9cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.025223 8550 ts_tablet_manager.cc:1929] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.025280 8550 log.cc:1199] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3209160f7b0e4f1a8c8121725454d088
I20251028 02:59:52.025516 8550 ts_tablet_manager.cc:1950] T 3209160f7b0e4f1a8c8121725454d088 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.026216 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3209160f7b0e4f1a8c8121725454d088 (table default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]) successfully deleted
W20251028 02:59:52.026625 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 8b88fda3f40540f3b42ee7f33dd33251 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b88fda3f40540f3b42ee7f33dd33251 already in progress: creating tablet
W20251028 02:59:52.027364 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 32ed23fd3cdc4be9ab0e65633c86e787 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32ed23fd3cdc4be9ab0e65633c86e787 already in progress: creating tablet
W20251028 02:59:52.027741 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a770dec97abb478b8c5504cc007b6092 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a770dec97abb478b8c5504cc007b6092 already in progress: creating tablet
I20251028 02:59:52.028205 8753 tablet.cc:1722] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:52.032765 8884 tablet.cc:1722] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:52.038185 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet cfd68d91fd9b4c69871383e90bd2ff85
I20251028 02:59:52.039124 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet cfd68d91fd9b4c69871383e90bd2ff85 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.039156 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet cfd68d91fd9b4c69871383e90bd2ff85 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.039183 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet cfd68d91fd9b4c69871383e90bd2ff85 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:52.039561 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet cfd68d91fd9b4c69871383e90bd2ff85 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cfd68d91fd9b4c69871383e90bd2ff85 already in progress: creating tablet
I20251028 02:59:52.039753 8148 raft_consensus.cc:3060] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.040783 8544 tablet_bootstrap.cc:492] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.040865 8544 ts_tablet_manager.cc:1403] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.000s
I20251028 02:59:52.040900 8545 tablet_bootstrap.cc:492] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.041016 8545 ts_tablet_manager.cc:1403] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.075s user 0.002s sys 0.000s
I20251028 02:59:52.041427 8544 raft_consensus.cc:359] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.041524 8544 raft_consensus.cc:385] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.041479 8545 raft_consensus.cc:359] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.041556 8544 raft_consensus.cc:740] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.041586 8545 raft_consensus.cc:385] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.041617 8545 raft_consensus.cc:740] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.041675 8544 consensus_queue.cc:260] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.041723 8545 consensus_queue.cc:260] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.041949 8544 ts_tablet_manager.cc:1434] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.041989 8545 ts_tablet_manager.cc:1434] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.042150 8545 tablet_bootstrap.cc:492] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.042157 8544 tablet_bootstrap.cc:492] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.043124 8545 tablet_bootstrap.cc:654] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.043198 8544 tablet_bootstrap.cc:654] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.043437 8553 raft_consensus.cc:493] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.043534 8553 raft_consensus.cc:515] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.043637 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_33402cf4a2024ce3b5e0f7471e82322c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:52.043949 8553 leader_election.cc:290] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 02:59:52.044054 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33402cf4a2024ce3b5e0f7471e82322c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.044725 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c24384fecd7b4c0a9b2e3da1e0369866" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.044842 8421 raft_consensus.cc:2468] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:52.045164 8220 leader_election.cc:304] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.045204 8546 tablet_bootstrap.cc:492] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.045301 8546 ts_tablet_manager.cc:1403] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.067s user 0.002s sys 0.000s
I20251028 02:59:52.045621 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c24384fecd7b4c0a9b2e3da1e0369866" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.045708 8545 tablet_bootstrap.cc:492] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.045782 8545 ts_tablet_manager.cc:1403] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251028 02:59:52.045781 8546 raft_consensus.cc:359] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.045910 8546 raft_consensus.cc:385] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.045948 8546 raft_consensus.cc:740] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
W20251028 02:59:52.046000 8218 leader_election.cc:343] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.046068 8546 consensus_queue.cc:260] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.046211 8556 raft_consensus.cc:2804] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.046254 8556 raft_consensus.cc:493] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.046291 8556 raft_consensus.cc:3060] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.046270 8545 raft_consensus.cc:359] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.046361 8545 raft_consensus.cc:385] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.046363 8546 ts_tablet_manager.cc:1434] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.046393 8545 raft_consensus.cc:740] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.046491 8545 consensus_queue.cc:260] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.046546 8546 tablet_bootstrap.cc:492] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.046777 8545 ts_tablet_manager.cc:1434] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:52.046947 8545 tablet_bootstrap.cc:492] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.047319 8556 raft_consensus.cc:515] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.047634 8546 tablet_bootstrap.cc:654] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.047683 8556 leader_election.cc:290] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:52.047927 8545 tablet_bootstrap.cc:654] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.048228 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c24384fecd7b4c0a9b2e3da1e0369866" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.048326 8421 raft_consensus.cc:3060] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:52.048470 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet cfd68d91fd9b4c69871383e90bd2ff85 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cfd68d91fd9b4c69871383e90bd2ff85 already in progress: creating tablet
I20251028 02:59:52.049263 8421 raft_consensus.cc:2468] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:52.049548 8220 leader_election.cc:304] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.049806 8553 raft_consensus.cc:2804] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.049886 8553 raft_consensus.cc:697] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:52.050024 8553 consensus_queue.cc:237] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.051390 8562 raft_consensus.cc:493] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.051482 8562 raft_consensus.cc:515] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.051854 8562 leader_election.cc:290] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.052996 8545 tablet_bootstrap.cc:492] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.053098 8545 ts_tablet_manager.cc:1403] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20251028 02:59:52.053248 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c24384fecd7b4c0a9b2e3da1e0369866" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.053570 8545 raft_consensus.cc:359] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.053694 8545 raft_consensus.cc:385] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.053735 8545 raft_consensus.cc:740] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
W20251028 02:59:52.053812 8218 leader_election.cc:343] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.053857 8545 consensus_queue.cc:260] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.054131 8545 ts_tablet_manager.cc:1434] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.054319 8545 tablet_bootstrap.cc:492] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:52.054611 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet cfd68d91fd9b4c69871383e90bd2ff85 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cfd68d91fd9b4c69871383e90bd2ff85 already in progress: creating tablet
I20251028 02:59:52.054879 8726 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:52.055377 8545 tablet_bootstrap.cc:654] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.055936 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_12396134528e4544a80934aa91d07353"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.056371 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12396134528e4544a80934aa91d07353 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:52.057412 8083 leader_election.cc:343] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: 6c6c40605a5a48d79837893443f4ca1b
W20251028 02:59:52.058085 8083 leader_election.cc:343] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 6c6c40605a5a48d79837893443f4ca1b
I20251028 02:59:52.058161 8083 leader_election.cc:304] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:52.059131 8912 raft_consensus.cc:2749] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:52.059959 8545 tablet_bootstrap.cc:492] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.060042 8545 ts_tablet_manager.cc:1403] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20251028 02:59:52.060231 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_fe2e8997c049473bb57e49992eeec9fa [id=7fe387bad5e740429e327d42f1efde22] alter complete (version 3)
I20251028 02:59:52.060554 8545 raft_consensus.cc:359] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.060637 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 482d7541b350426f9ff16448a24daeed
I20251028 02:59:52.060690 8545 raft_consensus.cc:385] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.060729 8545 raft_consensus.cc:740] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.057538 8546 tablet_bootstrap.cc:492] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.060840 8545 consensus_queue.cc:260] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.060892 8546 ts_tablet_manager.cc:1403] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20251028 02:59:52.060928 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 482d7541b350426f9ff16448a24daeed deleted and replaced by tablet 2d711941006d4417901135b7761a251f
I20251028 02:59:52.061120 8545 ts_tablet_manager.cc:1434] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.061650 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 9e908d5921d34c9396c1ccdc857c66b7 (table default.Table_a28e8e2125f541a6944d94f73ee01cae [id=4eb2743632814baab2e39bbdce700d6b]). Expected version 1 got 0
I20251028 02:59:52.061802 8545 tablet_bootstrap.cc:492] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.062266 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 482d7541b350426f9ff16448a24daeed with delete_type TABLET_DATA_DELETED (replaced by tablet 2d711941006d4417901135b7761a251f) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.062435 8552 tablet_replica.cc:333] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.062436 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_deda042aff7640a3afd2f724eba68912" } modify_external_catalogs: true
I20251028 02:59:52.062537 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_deda042aff7640a3afd2f724eba68912" } modify_external_catalogs: true
I20251028 02:59:52.062569 8552 raft_consensus.cc:2243] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.062688 8552 raft_consensus.cc:2272] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.062903 8545 tablet_bootstrap.cc:654] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.063457 8552 ts_tablet_manager.cc:1916] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.064416 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.064790 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.065604 8553 raft_consensus.cc:493] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.065685 8553 raft_consensus.cc:515] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.065918 8552 ts_tablet_manager.cc:1929] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.065976 8552 log.cc:1199] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/482d7541b350426f9ff16448a24daeed
I20251028 02:59:52.066114 8553 leader_election.cc:290] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:52.066231 8552 ts_tablet_manager.cc:1950] T 482d7541b350426f9ff16448a24daeed P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.066646 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b1b505f76ef84af6bc2b78a962bf6cf9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:52.066998 8218 leader_election.cc:343] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.067451 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 482d7541b350426f9ff16448a24daeed with delete_type TABLET_DATA_DELETED (replaced by tablet 2d711941006d4417901135b7761a251f) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.067622 8551 tablet_replica.cc:333] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.067659 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b1b505f76ef84af6bc2b78a962bf6cf9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.067760 8551 raft_consensus.cc:2243] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:52.067773 8421 raft_consensus.cc:2468] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:52.067978 8551 raft_consensus.cc:2272] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.068089 8220 leader_election.cc:304] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.068325 8553 raft_consensus.cc:2804] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.068377 8553 raft_consensus.cc:493] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.068418 8553 raft_consensus.cc:3060] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.068538 8551 ts_tablet_manager.cc:1916] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.069357 8553 raft_consensus.cc:515] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.069862 8553 leader_election.cc:290] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 02:59:52.070101 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 71feaa313a8f45798561741f7b5d9b20 consensus state has no leader
I20251028 02:59:52.070315 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e3e2db3778b049ce8d0327aa81f7e2ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:52.070389 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b1b505f76ef84af6bc2b78a962bf6cf9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.070492 8421 raft_consensus.cc:3060] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.070622 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b1b505f76ef84af6bc2b78a962bf6cf9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.070732 8551 ts_tablet_manager.cc:1929] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20251028 02:59:52.070739 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e3e2db3778b049ce8d0327aa81f7e2ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.070799 8551 log.cc:1199] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/482d7541b350426f9ff16448a24daeed
I20251028 02:59:52.071062 8551 ts_tablet_manager.cc:1950] T 482d7541b350426f9ff16448a24daeed P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.071415 8421 raft_consensus.cc:2468] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:52.071708 8220 leader_election.cc:304] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
W20251028 02:59:52.072018 8218 leader_election.cc:343] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.072291 8556 raft_consensus.cc:2804] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.072369 8556 raft_consensus.cc:697] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:52.072487 8556 consensus_queue.cc:237] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.072731 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fe2e8997c049473bb57e49992eeec9fa" } modify_external_catalogs: true
I20251028 02:59:52.072841 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fe2e8997c049473bb57e49992eeec9fa" } modify_external_catalogs: true
I20251028 02:59:52.073237 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_54486b99eed547a4aff3e091082b0b2f" } new_table_name: "default.Table_7778bd93ae56485f921a20d0df78a3ec" modify_external_catalogs: true
I20251028 02:59:52.073532 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 482d7541b350426f9ff16448a24daeed (table default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]) successfully deleted
I20251028 02:59:52.073734 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 482d7541b350426f9ff16448a24daeed (table default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]) successfully deleted
I20251028 02:59:52.073940 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d2e5ac67a4414c03b7890dea514caeee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.074479 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2e5ac67a4414c03b7890dea514caeee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.074978 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_114e1f461bc14e9aaa950f88bc67938e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.075342 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_114e1f461bc14e9aaa950f88bc67938e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.075727 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 482d7541b350426f9ff16448a24daeed with delete_type TABLET_DATA_DELETED (replaced by tablet 2d711941006d4417901135b7761a251f) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.075793 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c6c40605a5a48d79837893443f4ca1b with delete_type TABLET_DATA_DELETED (replaced by tablet aa1fe079ad014fe0b0d99efe7981de88) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.075942 8550 tablet_replica.cc:333] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.076086 8550 raft_consensus.cc:2243] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.076192 8550 raft_consensus.cc:2272] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.076501 8546 raft_consensus.cc:359] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.076802 8550 ts_tablet_manager.cc:1916] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.077383 8546 raft_consensus.cc:385] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.077437 8546 raft_consensus.cc:740] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.079102 8550 ts_tablet_manager.cc:1929] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.079173 8550 log.cc:1199] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/482d7541b350426f9ff16448a24daeed
I20251028 02:59:52.080396 8546 consensus_queue.cc:260] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.080358 7964 catalog_manager.cc:5649] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:52.080674 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 (table default.Table_54486b99eed547a4aff3e091082b0b2f [id=7a2a6ab3c9744942a9413c6cb057f527]). Expected version 3 got 0
I20251028 02:59:52.080749 8546 ts_tablet_manager.cc:1434] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.020s user 0.001s sys 0.000s
I20251028 02:59:52.081029 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet bc61d4b98fa34f8c9aa0656aa43bc25c
I20251028 02:59:52.081696 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet bc61d4b98fa34f8c9aa0656aa43bc25c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.082166 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet bc61d4b98fa34f8c9aa0656aa43bc25c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.082886 8648 raft_consensus.cc:493] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.082955 8550 ts_tablet_manager.cc:1950] T 482d7541b350426f9ff16448a24daeed P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.082999 8648 raft_consensus.cc:515] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.083067 8545 tablet_bootstrap.cc:492] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.083158 8545 ts_tablet_manager.cc:1403] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.001s
I20251028 02:59:52.083425 8648 leader_election.cc:290] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.083639 8550 tablet_replica.cc:333] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.083663 8545 raft_consensus.cc:359] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.083748 8545 raft_consensus.cc:385] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.083765 8550 raft_consensus.cc:2243] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.083779 8545 raft_consensus.cc:740] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.083858 8550 raft_consensus.cc:2272] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.083844 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet bc61d4b98fa34f8c9aa0656aa43bc25c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.083916 8545 consensus_queue.cc:260] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.084026 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63ee5aa5a6e841fdb9244563d75394ac" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.084230 8545 ts_tablet_manager.cc:1434] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.084422 8545 tablet_bootstrap.cc:492] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.084487 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63ee5aa5a6e841fdb9244563d75394ac" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.084592 8283 raft_consensus.cc:2468] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:52.084877 8356 leader_election.cc:304] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.085045 8549 raft_consensus.cc:2804] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.085103 8549 raft_consensus.cc:493] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.085140 8549 raft_consensus.cc:3060] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.085508 8545 tablet_bootstrap.cc:654] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:52.085614 8354 leader_election.cc:343] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.086119 8549 raft_consensus.cc:515] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.086483 8549 leader_election.cc:290] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.087553 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63ee5aa5a6e841fdb9244563d75394ac" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 02:59:52.087663 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 consensus state has no leader
I20251028 02:59:52.087678 8283 raft_consensus.cc:3060] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.088172 8550 ts_tablet_manager.cc:1916] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.088685 8283 raft_consensus.cc:2468] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
W20251028 02:59:52.089399 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet bc61d4b98fa34f8c9aa0656aa43bc25c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc61d4b98fa34f8c9aa0656aa43bc25c already in progress: creating tablet
I20251028 02:59:52.089450 8356 leader_election.cc:304] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.089612 8549 raft_consensus.cc:2804] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.089720 8549 raft_consensus.cc:697] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:52.089886 8549 consensus_queue.cc:237] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.090440 8550 ts_tablet_manager.cc:1929] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.090514 8550 log.cc:1199] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/6c6c40605a5a48d79837893443f4ca1b
I20251028 02:59:52.090729 8550 ts_tablet_manager.cc:1950] T 6c6c40605a5a48d79837893443f4ca1b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.092501 8544 tablet_bootstrap.cc:492] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.092612 8544 ts_tablet_manager.cc:1403] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.000s
I20251028 02:59:52.092746 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63ee5aa5a6e841fdb9244563d75394ac" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.093107 8544 raft_consensus.cc:359] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.093211 8544 raft_consensus.cc:385] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.093192 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 1c81698dcbe14d5ba61471e7249705db (table default.Table_310089f9c2b34a7e9e1c01c97fdb9b11 [id=c103fff3458d4d08affefb7c38d35e52]). Expected version 1 got 0
I20251028 02:59:52.093248 8544 raft_consensus.cc:740] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.093389 8544 consensus_queue.cc:260] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.093743 8544 ts_tablet_manager.cc:1434] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.093947 8544 tablet_bootstrap.cc:492] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.094082 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 482d7541b350426f9ff16448a24daeed (table default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]) successfully deleted
I20251028 02:59:52.094223 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 6c6c40605a5a48d79837893443f4ca1b (table default.Table_dff2d248a846455eb7bd3f3648f7ceee [id=77dd1eb6df0747bcab57b157ab8a850e]) successfully deleted
I20251028 02:59:52.094376 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e5f0326a03040de81a506caa80dcbd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.094376 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 482d7541b350426f9ff16448a24daeed with delete_type TABLET_DATA_DELETED (replaced by tablet 2d711941006d4417901135b7761a251f) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:52.094616 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet bc61d4b98fa34f8c9aa0656aa43bc25c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc61d4b98fa34f8c9aa0656aa43bc25c already in progress: creating tablet
W20251028 02:59:52.095021 8354 leader_election.cc:343] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.095162 8544 tablet_bootstrap.cc:654] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.096149 8546 tablet_bootstrap.cc:492] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.096423 8552 tablet_replica.cc:333] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.096570 8552 raft_consensus.cc:2243] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.096673 8552 raft_consensus.cc:2272] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.096856 8545 tablet_bootstrap.cc:492] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.096946 8545 ts_tablet_manager.cc:1403] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20251028 02:59:52.097174 8552 ts_tablet_manager.cc:1916] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.097414 8545 raft_consensus.cc:359] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.097520 8545 raft_consensus.cc:385] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.097551 8545 raft_consensus.cc:740] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.097661 8545 consensus_queue.cc:260] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.097935 8545 ts_tablet_manager.cc:1434] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.098111 8545 tablet_bootstrap.cc:492] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.099081 8545 tablet_bootstrap.cc:654] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.099244 8544 tablet_bootstrap.cc:492] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.099352 8544 ts_tablet_manager.cc:1403] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:52.099848 8544 raft_consensus.cc:359] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.099959 8544 raft_consensus.cc:385] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.100006 8544 raft_consensus.cc:740] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.100143 8544 consensus_queue.cc:260] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.100385 8544 ts_tablet_manager.cc:1434] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.100570 8544 tablet_bootstrap.cc:492] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.100859 8552 ts_tablet_manager.cc:1929] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.100936 8552 log.cc:1199] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/bc61d4b98fa34f8c9aa0656aa43bc25c
I20251028 02:59:52.101171 8552 ts_tablet_manager.cc:1950] T bc61d4b98fa34f8c9aa0656aa43bc25c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.101662 8544 tablet_bootstrap.cc:654] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.102236 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f4bebc4575e4462cbc8a02fca4fb228b
I20251028 02:59:52.103605 8546 tablet_bootstrap.cc:654] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.105198 8545 tablet_bootstrap.cc:492] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.105298 8545 ts_tablet_manager.cc:1403] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:52.105803 8545 raft_consensus.cc:359] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.105934 8545 raft_consensus.cc:385] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.105966 8545 raft_consensus.cc:740] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.106073 8545 consensus_queue.cc:260] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.106335 8545 ts_tablet_manager.cc:1434] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.106490 8545 tablet_bootstrap.cc:492] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.107288 8552 tablet_replica.cc:333] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.107395 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 482d7541b350426f9ff16448a24daeed with delete_type TABLET_DATA_DELETED (replaced by tablet 2d711941006d4417901135b7761a251f) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.107571 8552 raft_consensus.cc:2243] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.107769 8552 raft_consensus.cc:2272] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.108659 8552 ts_tablet_manager.cc:1916] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.108899 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 11a8056e97ff452e93e91622fedb77a6
I20251028 02:59:52.109186 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet f4bebc4575e4462cbc8a02fca4fb228b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.109974 8544 tablet_bootstrap.cc:492] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.110122 8544 ts_tablet_manager.cc:1403] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20251028 02:59:52.101858 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet bc61d4b98fa34f8c9aa0656aa43bc25c (table default.Table_4c1aacffea564931bb49e6bc29c6b0b7 [id=449d979448934b43a49f0b431b6dc45c]) successfully deleted
I20251028 02:59:52.111025 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e5f0326a03040de81a506caa80dcbd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.111168 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 482d7541b350426f9ff16448a24daeed with delete_type TABLET_DATA_DELETED (replaced by tablet 2d711941006d4417901135b7761a251f) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.111284 8551 tablet_replica.cc:333] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.111343 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet f4bebc4575e4462cbc8a02fca4fb228b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.111418 8551 raft_consensus.cc:2243] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.111517 8551 raft_consensus.cc:2272] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.111488 8544 raft_consensus.cc:359] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.111588 8544 raft_consensus.cc:385] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.111624 8544 raft_consensus.cc:740] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.111946 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 11a8056e97ff452e93e91622fedb77a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.112030 8544 consensus_queue.cc:260] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.112129 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 11a8056e97ff452e93e91622fedb77a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.112201 8551 ts_tablet_manager.cc:1916] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.112331 8544 ts_tablet_manager.cc:1434] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 02:59:52.112471 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 482d7541b350426f9ff16448a24daeed because the tablet was not found. No further retry: Not found: Tablet not found: 482d7541b350426f9ff16448a24daeed
I20251028 02:59:52.112524 8544 tablet_bootstrap.cc:492] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.112625 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet f4bebc4575e4462cbc8a02fca4fb228b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:52.112636 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f4bebc4575e4462cbc8a02fca4fb228b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4bebc4575e4462cbc8a02fca4fb228b already in progress: creating tablet
I20251028 02:59:52.113534 8544 tablet_bootstrap.cc:654] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.114614 8551 ts_tablet_manager.cc:1929] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.114692 8551 log.cc:1199] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3e5f0326a03040de81a506caa80dcbd9
I20251028 02:59:52.114866 8544 tablet_bootstrap.cc:492] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.114929 8551 ts_tablet_manager.cc:1950] T 3e5f0326a03040de81a506caa80dcbd9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.114956 8544 ts_tablet_manager.cc:1403] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:52.115453 8544 raft_consensus.cc:359] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.115556 8544 raft_consensus.cc:385] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.115592 8544 raft_consensus.cc:740] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.115638 8551 tablet_replica.cc:333] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.115711 8544 consensus_queue.cc:260] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.115747 8551 raft_consensus.cc:2243] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.115849 8551 raft_consensus.cc:2272] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.115983 8544 ts_tablet_manager.cc:1434] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.115999 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 11a8056e97ff452e93e91622fedb77a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.116127 8550 tablet_replica.cc:333] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.116163 8544 tablet_bootstrap.cc:492] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.116215 8550 raft_consensus.cc:2243] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.116253 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3e5f0326a03040de81a506caa80dcbd9 (table default.Table_c320b509540344b7895fb967b844a408 [id=fbecd1cf849b47bb880b96a56333fbd9]) successfully deleted
I20251028 02:59:52.116298 8550 raft_consensus.cc:2272] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:52.116418 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 482d7541b350426f9ff16448a24daeed because the tablet was not found. No further retry: Not found: Tablet not found: 482d7541b350426f9ff16448a24daeed
I20251028 02:59:52.116784 8545 tablet_bootstrap.cc:654] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.117159 8544 tablet_bootstrap.cc:654] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.117750 8551 ts_tablet_manager.cc:1916] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.118255 8550 ts_tablet_manager.cc:1916] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.119099 8552 ts_tablet_manager.cc:1929] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.119899 8551 ts_tablet_manager.cc:1929] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.119971 8551 log.cc:1199] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/f4bebc4575e4462cbc8a02fca4fb228b
I20251028 02:59:52.120168 8551 ts_tablet_manager.cc:1950] T f4bebc4575e4462cbc8a02fca4fb228b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.120429 8550 ts_tablet_manager.cc:1929] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.120486 8550 log.cc:1199] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/11a8056e97ff452e93e91622fedb77a6
I20251028 02:59:52.120666 8550 ts_tablet_manager.cc:1950] T 11a8056e97ff452e93e91622fedb77a6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.120842 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet f4bebc4575e4462cbc8a02fca4fb228b (table default.Table_deda042aff7640a3afd2f724eba68912 [id=e7c4b52cbf8c496e94ca8634d53e41de]) successfully deleted
I20251028 02:59:52.121070 8551 tablet_replica.cc:333] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.121201 8551 raft_consensus.cc:2243] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:52.121416 8551 raft_consensus.cc:2272] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.122207 8544 tablet_bootstrap.cc:492] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.122295 8544 ts_tablet_manager.cc:1403] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:52.122319 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet c5971fbd47e64ffda6df61dfc4756dfb with delete_type TABLET_DATA_DELETED (replaced by tablet 6f38ce803e114831ace1412abdad7126) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.122483 8550 tablet_replica.cc:333] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.122584 8550 raft_consensus.cc:2243] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.122820 8544 raft_consensus.cc:359] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.122922 8544 raft_consensus.cc:385] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.122953 8544 raft_consensus.cc:740] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.123010 8545 tablet_bootstrap.cc:492] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.123082 8545 ts_tablet_manager.cc:1403] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 02:59:52.123083 8544 consensus_queue.cc:260] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.123378 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 374936c130c841348e0d76cce070a295 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.123548 8550 raft_consensus.cc:2272] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.123607 8545 raft_consensus.cc:359] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.123708 8545 raft_consensus.cc:385] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.123744 8545 raft_consensus.cc:740] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.123811 8544 ts_tablet_manager.cc:1434] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.123869 8545 consensus_queue.cc:260] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.123982 8544 tablet_bootstrap.cc:492] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.124763 8545 ts_tablet_manager.cc:1434] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:52.124956 8545 tablet_bootstrap.cc:492] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.124959 8552 log.cc:1199] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3e5f0326a03040de81a506caa80dcbd9
I20251028 02:59:52.124993 8544 tablet_bootstrap.cc:654] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.125234 8552 ts_tablet_manager.cc:1950] T 3e5f0326a03040de81a506caa80dcbd9 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.125926 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3e5f0326a03040de81a506caa80dcbd9 (table default.Table_c320b509540344b7895fb967b844a408 [id=fbecd1cf849b47bb880b96a56333fbd9]) successfully deleted
I20251028 02:59:52.126022 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 6d94d536c1054613b2ae8ffcb08fbaab (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 1 got 0
I20251028 02:59:52.126266 8545 tablet_bootstrap.cc:654] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.126134 7962 catalog_manager.cc:5649] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:52.126425 7962 catalog_manager.cc:5649] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:52.126237 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 11a8056e97ff452e93e91622fedb77a6 (table default.Table_fe2e8997c049473bb57e49992eeec9fa [id=7fe387bad5e740429e327d42f1efde22]) successfully deleted
I20251028 02:59:52.127018 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 2be7be7c732e4780859cbe6624edf54e (DEFAULT_TABLE table=default.Table_2a09149ed84d4343a81458e2fb78f262 [id=2bcc83cc472b4ec5a7ca79a8a7c5ff23]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.127318 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2be7be7c732e4780859cbe6624edf54e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.127682 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 2be7be7c732e4780859cbe6624edf54e (DEFAULT_TABLE table=default.Table_2a09149ed84d4343a81458e2fb78f262 [id=2bcc83cc472b4ec5a7ca79a8a7c5ff23]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.127888 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2be7be7c732e4780859cbe6624edf54e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.128603 8552 tablet_replica.cc:333] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.128727 8552 raft_consensus.cc:2243] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.128854 8552 raft_consensus.cc:2272] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.129582 8648 raft_consensus.cc:493] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.129665 8648 raft_consensus.cc:515] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.130090 8648 leader_election.cc:290] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.130579 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6d94d536c1054613b2ae8ffcb08fbaab" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.130704 8283 raft_consensus.cc:2468] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:52.130990 8356 leader_election.cc:304] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.131356 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 (DEFAULT_TABLE table=default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.131597 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.131698 8544 tablet_bootstrap.cc:492] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.131768 8544 ts_tablet_manager.cc:1403] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:52.132248 8544 raft_consensus.cc:359] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.132339 8544 raft_consensus.cc:385] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.132388 8544 raft_consensus.cc:740] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.132515 8544 consensus_queue.cc:260] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.132831 8544 ts_tablet_manager.cc:1434] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.133332 8544 tablet_bootstrap.cc:492] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.133901 8648 raft_consensus.cc:2804] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.133967 8648 raft_consensus.cc:493] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.134007 8648 raft_consensus.cc:3060] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.134290 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 (table default.Table_7778bd93ae56485f921a20d0df78a3ec [id=7a2a6ab3c9744942a9413c6cb057f527]). Expected version 4 got 0
I20251028 02:59:52.134377 8544 tablet_bootstrap.cc:654] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.134387 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 6d94d536c1054613b2ae8ffcb08fbaab (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 1 got 0
I20251028 02:59:52.134464 7968 catalog_manager.cc:5649] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:52.134843 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6d94d536c1054613b2ae8ffcb08fbaab" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:52.135023 7962 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet 6d94d536c1054613b2ae8ffcb08fbaab (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 6d94d536c1054613b2ae8ffcb08fbaab consensus state has no leader
I20251028 02:59:52.135056 8648 raft_consensus.cc:515] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.135453 8648 leader_election.cc:290] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.136004 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 3057f4749edd43ac889f64d1f4e282d1 (DEFAULT_TABLE table=default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.136044 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6d94d536c1054613b2ae8ffcb08fbaab" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:52.136160 8283 raft_consensus.cc:3060] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.137113 8283 raft_consensus.cc:2468] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:52.137118 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 025a4d3e71ff443c8bdf649055a17d54 (table default.Table_97b3a76d004f47c981178cf580a95985 [id=7d237f6c141e4d5e821305500a020dc0]). Expected version 2 got 0
I20251028 02:59:52.137146 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 9b92443ae55f4a489d92d6175b03f377 (DEFAULT_TABLE table=default.Table_2ca85f32e093464c8bfade073ee1e7c3 [id=a6d01f64a2e34c5781f1e090b1d2cd88]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.137463 8356 leader_election.cc:304] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.137976 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 11a8056e97ff452e93e91622fedb77a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:52.138116 7955 catalog_manager.cc:4952] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 482d7541b350426f9ff16448a24daeed because the tablet was not found. No further retry: Not found: Tablet not found: 482d7541b350426f9ff16448a24daeed
I20251028 02:59:52.138744 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 9b92443ae55f4a489d92d6175b03f377 (DEFAULT_TABLE table=default.Table_2ca85f32e093464c8bfade073ee1e7c3 [id=a6d01f64a2e34c5781f1e090b1d2cd88]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.138927 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 3057f4749edd43ac889f64d1f4e282d1 (DEFAULT_TABLE table=default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.139005 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b92443ae55f4a489d92d6175b03f377. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.139118 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3057f4749edd43ac889f64d1f4e282d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.140537 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_596fa65222fb43afbb4d4b620ee0a0c1" } new_table_name: "default.Table_b237c0d768794e3592deda86eb2a4945" modify_external_catalogs: true
I20251028 02:59:52.140981 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 909b08e0860b4e4f9bb1822acfd1dc02 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.141697 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 2be7be7c732e4780859cbe6624edf54e (DEFAULT_TABLE table=default.Table_2a09149ed84d4343a81458e2fb78f262 [id=2bcc83cc472b4ec5a7ca79a8a7c5ff23]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.141772 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 (DEFAULT_TABLE table=default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.141973 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2be7be7c732e4780859cbe6624edf54e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.142001 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 9b92443ae55f4a489d92d6175b03f377 (DEFAULT_TABLE table=default.Table_2ca85f32e093464c8bfade073ee1e7c3 [id=a6d01f64a2e34c5781f1e090b1d2cd88]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:52.142328 8354 leader_election.cc:343] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.143174 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet bc61d4b98fa34f8c9aa0656aa43bc25c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.143800 8553 raft_consensus.cc:493] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.143913 8553 raft_consensus.cc:515] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.144181 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 3057f4749edd43ac889f64d1f4e282d1 (DEFAULT_TABLE table=default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.144291 8553 leader_election.cc:290] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.144721 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6d94d536c1054613b2ae8ffcb08fbaab" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.144778 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1c81698dcbe14d5ba61471e7249705db" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.144877 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b92443ae55f4a489d92d6175b03f377. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.146385 8648 raft_consensus.cc:2804] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.146476 8648 raft_consensus.cc:697] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:52.146611 8648 consensus_queue.cc:237] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.146781 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1c81698dcbe14d5ba61471e7249705db" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.146871 8421 raft_consensus.cc:2468] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:52.147168 8220 leader_election.cc:304] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.147360 8553 raft_consensus.cc:2804] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.147416 8553 raft_consensus.cc:493] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.147459 8553 raft_consensus.cc:3060] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.147552 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3057f4749edd43ac889f64d1f4e282d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.147491 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 (DEFAULT_TABLE table=default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.147744 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.148414 8553 raft_consensus.cc:515] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.148788 8553 leader_election.cc:290] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.148824 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet f11c46ed489e41d3a25cfcd9413443af with delete_type TABLET_DATA_DELETED (replaced by tablet 9f1c1e14665c4712bffa439ab98b2a6e) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.149808 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b92443ae55f4a489d92d6175b03f377. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.143676 8552 ts_tablet_manager.cc:1916] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.151182 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 32ed23fd3cdc4be9ab0e65633c86e787 with delete_type TABLET_DATA_DELETED (replaced by tablet 2be7be7c732e4780859cbe6624edf54e) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.151697 8551 ts_tablet_manager.cc:1916] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.151782 8546 tablet_bootstrap.cc:492] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.151878 8546 ts_tablet_manager.cc:1403] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.056s user 0.001s sys 0.001s
I20251028 02:59:52.152125 8550 ts_tablet_manager.cc:1916] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.152390 8546 raft_consensus.cc:359] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.152491 8546 raft_consensus.cc:385] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.152532 8546 raft_consensus.cc:740] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.152652 8546 consensus_queue.cc:260] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.152704 8260 tablet_service.cc:1505] Processing CreateTablet for tablet f57b4fa3ceec4557bc6d6a2125d80790 (DEFAULT_TABLE table=default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.152866 8394 tablet_service.cc:1505] Processing CreateTablet for tablet f57b4fa3ceec4557bc6d6a2125d80790 (DEFAULT_TABLE table=default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.152954 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f57b4fa3ceec4557bc6d6a2125d80790. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.152976 8546 ts_tablet_manager.cc:1434] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:52.153184 8546 tablet_bootstrap.cc:492] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.153954 8400 tablet_service.cc:1505] Processing CreateTablet for tablet e359f0121c334cc1a8b3eedbbe64bda2 (DEFAULT_TABLE table=default.Table_013e772cd60943509e2ea1c830dfe151 [id=6630b83265864f36ad74cac692f131e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.154106 8550 ts_tablet_manager.cc:1929] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.154172 8550 log.cc:1199] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c5971fbd47e64ffda6df61dfc4756dfb
I20251028 02:59:52.154213 8262 tablet_service.cc:1505] Processing CreateTablet for tablet e359f0121c334cc1a8b3eedbbe64bda2 (DEFAULT_TABLE table=default.Table_013e772cd60943509e2ea1c830dfe151 [id=6630b83265864f36ad74cac692f131e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.154379 8550 ts_tablet_manager.cc:1950] T c5971fbd47e64ffda6df61dfc4756dfb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.154446 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e359f0121c334cc1a8b3eedbbe64bda2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.154924 8550 tablet_replica.cc:333] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.155035 8550 raft_consensus.cc:2243] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.155133 8550 raft_consensus.cc:2272] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:52.156358 8354 leader_election.cc:343] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.156489 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1c81698dcbe14d5ba61471e7249705db" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:52.156555 8218 leader_election.cc:343] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.156762 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet c5971fbd47e64ffda6df61dfc4756dfb (table default.Table_828e8df9af6a48168a189935f63b2941 [id=42c89dfccc714a9eb42e3779130c8af4]) successfully deleted
I20251028 02:59:52.157454 8546 tablet_bootstrap.cc:654] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:52.157727 8218 leader_election.cc:343] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.159030 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1c81698dcbe14d5ba61471e7249705db" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.159153 8421 raft_consensus.cc:3060] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.160079 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 32ed23fd3cdc4be9ab0e65633c86e787 with delete_type TABLET_DATA_DELETED (replaced by tablet 2be7be7c732e4780859cbe6624edf54e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.160147 8421 raft_consensus.cc:2468] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:52.160427 8549 raft_consensus.cc:493] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.160513 8549 raft_consensus.cc:515] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.160813 8549 leader_election.cc:290] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.161151 8220 leader_election.cc:304] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.161259 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f57b4fa3ceec4557bc6d6a2125d80790. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.161232 8116 tablet_service.cc:1505] Processing CreateTablet for tablet f57b4fa3ceec4557bc6d6a2125d80790 (DEFAULT_TABLE table=default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.161521 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f57b4fa3ceec4557bc6d6a2125d80790. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.162457 8553 raft_consensus.cc:2804] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.162555 8553 raft_consensus.cc:697] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:52.162681 8553 consensus_queue.cc:237] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.162928 8118 tablet_service.cc:1505] Processing CreateTablet for tablet e359f0121c334cc1a8b3eedbbe64bda2 (DEFAULT_TABLE table=default.Table_013e772cd60943509e2ea1c830dfe151 [id=6630b83265864f36ad74cac692f131e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.163163 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e359f0121c334cc1a8b3eedbbe64bda2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.163635 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3057f4749edd43ac889f64d1f4e282d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.165050 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "025a4d3e71ff443c8bdf649055a17d54" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.165172 8285 raft_consensus.cc:2468] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:52.165459 8356 leader_election.cc:304] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.165874 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "025a4d3e71ff443c8bdf649055a17d54" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.165979 8148 raft_consensus.cc:2468] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:52.166239 8549 raft_consensus.cc:2804] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.166297 8549 raft_consensus.cc:493] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.166334 8549 raft_consensus.cc:3060] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.166765 8753 tablet.cc:1722] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.167037 8551 ts_tablet_manager.cc:1929] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:52.167109 8551 log.cc:1199] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/11a8056e97ff452e93e91622fedb77a6
I20251028 02:59:52.167158 8549 raft_consensus.cc:515] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.166246 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.167394 8551 ts_tablet_manager.cc:1950] T 11a8056e97ff452e93e91622fedb77a6 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.167930 8551 tablet_replica.cc:333] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.168053 8551 raft_consensus.cc:2243] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.168152 8551 raft_consensus.cc:2272] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.168625 8551 ts_tablet_manager.cc:1916] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.170475 8549 leader_election.cc:290] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.171109 8545 tablet_bootstrap.cc:492] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.171190 8545 ts_tablet_manager.cc:1403] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.000s
I20251028 02:59:52.171705 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 11a8056e97ff452e93e91622fedb77a6 (table default.Table_fe2e8997c049473bb57e49992eeec9fa [id=7fe387bad5e740429e327d42f1efde22]) successfully deleted
I20251028 02:59:52.171905 8556 raft_consensus.cc:493] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.171998 8556 raft_consensus.cc:515] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.172029 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "025a4d3e71ff443c8bdf649055a17d54" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:52.172145 8285 raft_consensus.cc:3060] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.172272 8545 raft_consensus.cc:359] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.172363 8545 raft_consensus.cc:385] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.172367 8556 leader_election.cc:290] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.172394 8545 raft_consensus.cc:740] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.172340 8726 tablet.cc:1722] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.172502 8545 consensus_queue.cc:260] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.172508 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "025a4d3e71ff443c8bdf649055a17d54" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.172603 8146 raft_consensus.cc:3060] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.172757 8545 ts_tablet_manager.cc:1434] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 02:59:52.172945 8545 tablet_bootstrap.cc:492] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.173121 8285 raft_consensus.cc:2468] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:52.173568 8146 raft_consensus.cc:2468] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:52.173642 8884 tablet.cc:1722] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.174094 8545 tablet_bootstrap.cc:654] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.170475 8551 ts_tablet_manager.cc:1929] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.174907 8551 log.cc:1199] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/bc61d4b98fa34f8c9aa0656aa43bc25c
I20251028 02:59:52.175274 8551 ts_tablet_manager.cc:1950] T bc61d4b98fa34f8c9aa0656aa43bc25c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.175920 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet bc61d4b98fa34f8c9aa0656aa43bc25c (table default.Table_4c1aacffea564931bb49e6bc29c6b0b7 [id=449d979448934b43a49f0b431b6dc45c]) successfully deleted
I20251028 02:59:52.176115 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e359f0121c334cc1a8b3eedbbe64bda2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.176891 8544 tablet_bootstrap.cc:492] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.176976 8544 ts_tablet_manager.cc:1403] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.000s
I20251028 02:59:52.178018 8544 raft_consensus.cc:359] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.178129 8544 raft_consensus.cc:385] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.178171 8544 raft_consensus.cc:740] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.178304 8356 leader_election.cc:304] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.178545 8549 raft_consensus.cc:2804] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.178685 8549 raft_consensus.cc:697] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:52.178812 8549 consensus_queue.cc:237] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.179637 8556 raft_consensus.cc:493] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.178786 8550 ts_tablet_manager.cc:1916] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.179723 8556 raft_consensus.cc:515] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.180099 8556 leader_election.cc:290] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.180567 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e3cbb0c3d914ffebfd2c976ce8b1098" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.180680 8421 raft_consensus.cc:2468] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
W20251028 02:59:52.181188 8220 leader_election.cc:343] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:52.181352 8220 leader_election.cc:304] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.181753 8556 raft_consensus.cc:2804] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.181809 8556 raft_consensus.cc:493] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.181874 8556 raft_consensus.cc:3060] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.181876 8550 ts_tablet_manager.cc:1929] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.181946 8550 log.cc:1199] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/374936c130c841348e0d76cce070a295
I20251028 02:59:52.182168 8550 ts_tablet_manager.cc:1950] T 374936c130c841348e0d76cce070a295 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.182790 8556 raft_consensus.cc:515] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.183174 8556 leader_election.cc:290] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.183748 8551 tablet_replica.cc:333] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.183784 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e3cbb0c3d914ffebfd2c976ce8b1098" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.183774 8544 consensus_queue.cc:260] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.183894 8551 raft_consensus.cc:2243] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.183995 8551 raft_consensus.cc:2272] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.183897 8420 raft_consensus.cc:3060] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.184427 8544 ts_tablet_manager.cc:1434] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:52.184561 8551 ts_tablet_manager.cc:1916] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.184612 8544 tablet_bootstrap.cc:492] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.185357 8420 raft_consensus.cc:2468] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:52.185691 8220 leader_election.cc:304] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.186146 8556 raft_consensus.cc:2804] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.186228 8556 raft_consensus.cc:697] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:52.186355 8556 consensus_queue.cc:237] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.186594 8544 tablet_bootstrap.cc:654] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.186662 8551 ts_tablet_manager.cc:1929] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.186722 8551 log.cc:1199] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/32ed23fd3cdc4be9ab0e65633c86e787
I20251028 02:59:52.186933 8551 ts_tablet_manager.cc:1950] T 32ed23fd3cdc4be9ab0e65633c86e787 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.187114 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 374936c130c841348e0d76cce070a295 (table default.Table_bb2a8beb93a74d8eb05afe4fb154bbfa [id=90264f1fa3994b8a944e512a95361ff2]) successfully deleted
I20251028 02:59:52.187253 8550 tablet_replica.cc:333] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.187398 8550 raft_consensus.cc:2243] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:52.187402 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 11a8056e97ff452e93e91622fedb77a6 because the tablet was not found. No further retry: Not found: Tablet not found: 11a8056e97ff452e93e91622fedb77a6
I20251028 02:59:52.187496 8550 raft_consensus.cc:2272] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.187603 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 32ed23fd3cdc4be9ab0e65633c86e787 (table default.Table_2a09149ed84d4343a81458e2fb78f262 [id=2bcc83cc472b4ec5a7ca79a8a7c5ff23]) successfully deleted
I20251028 02:59:52.188117 8550 ts_tablet_manager.cc:1916] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.188372 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2fb1b38dc3684e8b9471ec10146a1a3d" } new_table_name: "default.Table_7f5ff2cc8eef4fa28d84a14d3851d912" modify_external_catalogs: true
I20251028 02:59:52.188952 8544 tablet_bootstrap.cc:492] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.189029 8544 ts_tablet_manager.cc:1403] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:52.189144 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_b237c0d768794e3592deda86eb2a4945 [id=4a127e52978a4c8ea8a947478ce5996d] alter complete (version 2)
I20251028 02:59:52.189517 8544 raft_consensus.cc:359] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.189628 8544 raft_consensus.cc:385] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.189661 8544 raft_consensus.cc:740] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.189811 8544 consensus_queue.cc:260] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.190065 8544 ts_tablet_manager.cc:1434] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.190635 8544 tablet_bootstrap.cc:492] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.191663 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e3cbb0c3d914ffebfd2c976ce8b1098" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.191870 8550 ts_tablet_manager.cc:1929] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.191936 8550 log.cc:1199] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/f11c46ed489e41d3a25cfcd9413443af
I20251028 02:59:52.192013 8544 tablet_bootstrap.cc:654] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.192155 8550 ts_tablet_manager.cc:1950] T f11c46ed489e41d3a25cfcd9413443af P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.192782 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet f11c46ed489e41d3a25cfcd9413443af (table default.Table_ec3580a0e3af454f9154d6733e07ca55 [id=3eb0b70129be40e598874d492735ba51]) successfully deleted
W20251028 02:59:52.193011 8218 leader_election.cc:343] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.193324 8544 tablet_bootstrap.cc:492] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.193398 8544 ts_tablet_manager.cc:1403] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:52.193895 8544 raft_consensus.cc:359] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.193990 8544 raft_consensus.cc:385] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.194020 8544 raft_consensus.cc:740] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.194137 8544 consensus_queue.cc:260] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:52.194306 8218 leader_election.cc:343] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:52.194387 8544 ts_tablet_manager.cc:1434] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.194387 8218 leader_election.cc:304] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.194449 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3e3cbb0c3d914ffebfd2c976ce8b1098" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.194604 8544 tablet_bootstrap.cc:492] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.194607 8553 raft_consensus.cc:2749] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:52.194964 8545 tablet_bootstrap.cc:492] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.195068 8545 ts_tablet_manager.cc:1403] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20251028 02:59:52.195628 8544 tablet_bootstrap.cc:654] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.195648 8545 raft_consensus.cc:359] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.195747 8545 raft_consensus.cc:385] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.195784 8545 raft_consensus.cc:740] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.195932 8545 consensus_queue.cc:260] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.196234 8545 ts_tablet_manager.cc:1434] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:52.196437 8545 tablet_bootstrap.cc:492] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:52.196578 8218 leader_election.cc:343] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.197458 8545 tablet_bootstrap.cc:654] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.198135 8552 ts_tablet_manager.cc:1929] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:52.198207 8552 log.cc:1199] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/11a8056e97ff452e93e91622fedb77a6
I20251028 02:59:52.198493 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_60b178c0b65e441d9b23d8b4bc53dedb" } new_table_name: "default.Table_0424d53c61864c5cbaeb7c12b0b6d96c" modify_external_catalogs: true
I20251028 02:59:52.198613 8552 ts_tablet_manager.cc:1950] T 11a8056e97ff452e93e91622fedb77a6 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.199343 8552 tablet_replica.cc:333] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.199551 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 11a8056e97ff452e93e91622fedb77a6 (table default.Table_fe2e8997c049473bb57e49992eeec9fa [id=7fe387bad5e740429e327d42f1efde22]) successfully deleted
I20251028 02:59:52.200325 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 66fb0173e40e40999f90f015ddb294b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.200491 8550 tablet_replica.cc:333] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.200621 8550 raft_consensus.cc:2243] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.200660 8552 raft_consensus.cc:2243] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.200726 8550 raft_consensus.cc:2272] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.200779 8552 raft_consensus.cc:2272] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.201236 8552 ts_tablet_manager.cc:1916] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.201524 7968 catalog_manager.cc:5649] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:52.201757 7968 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 1c81698dcbe14d5ba61471e7249705db (table default.Table_310089f9c2b34a7e9e1c01c97fdb9b11 [id=c103fff3458d4d08affefb7c38d35e52]). Expected version 1 got 0
I20251028 02:59:52.202947 7969 catalog_manager.cc:5649] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:52.203137 7969 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 6d94d536c1054613b2ae8ffcb08fbaab (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 1 got 0
I20251028 02:59:52.203917 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 9c1d8552cc1941bda2e9f02fba56c9f9 (DEFAULT_TABLE table=default.Table_114e1f461bc14e9aaa950f88bc67938e [id=4e8c5072da4e4ec6a616ba4c7fc91cd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.204208 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1d8552cc1941bda2e9f02fba56c9f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.204447 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_afc8d935c16a4f0b9d3bff7efa913adf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.204785 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_afc8d935c16a4f0b9d3bff7efa913adf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:52.205644 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0703e18d7dba4d628e3cc310be55fcf5 consensus state has no leader
I20251028 02:59:52.205696 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 9c1d8552cc1941bda2e9f02fba56c9f9 (DEFAULT_TABLE table=default.Table_114e1f461bc14e9aaa950f88bc67938e [id=4e8c5072da4e4ec6a616ba4c7fc91cd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.205991 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1d8552cc1941bda2e9f02fba56c9f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.207412 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 76f4055230904a25a1241a0a86a059d2 (DEFAULT_TABLE table=default.Table_d2e5ac67a4414c03b7890dea514caeee [id=c8f1a159643642eb8dc2f202d73532fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.207438 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_af395b1589f74f4ebe9dd06515819408"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:52.207648 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76f4055230904a25a1241a0a86a059d2. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:52.207793 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af395b1589f74f4ebe9dd06515819408 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.207893 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b9407dac1631402082afb2b12dc61dbd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.208213 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9407dac1631402082afb2b12dc61dbd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.209208 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2a09149ed84d4343a81458e2fb78f262" } modify_external_catalogs: true
I20251028 02:59:52.209302 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2a09149ed84d4343a81458e2fb78f262" } modify_external_catalogs: true
I20251028 02:59:52.211093 8912 consensus_queue.cc:1048] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.212353 8550 ts_tablet_manager.cc:1916] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.214422 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet d928418826a84f63803ae474dc05de7e from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.214641 8550 ts_tablet_manager.cc:1929] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.214706 8550 log.cc:1199] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/66fb0173e40e40999f90f015ddb294b7
I20251028 02:59:52.214932 8550 ts_tablet_manager.cc:1950] T 66fb0173e40e40999f90f015ddb294b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.215588 8545 tablet_bootstrap.cc:492] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.215679 8545 ts_tablet_manager.cc:1403] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20251028 02:59:52.216192 8545 raft_consensus.cc:359] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.216296 8545 raft_consensus.cc:385] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.216331 8545 raft_consensus.cc:740] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.216442 8545 consensus_queue.cc:260] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.216747 8545 ts_tablet_manager.cc:1434] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.216913 8545 tablet_bootstrap.cc:492] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.217145 7969 catalog_manager.cc:5649] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:52.217350 7969 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 025a4d3e71ff443c8bdf649055a17d54 (table default.Table_97b3a76d004f47c981178cf580a95985 [id=7d237f6c141e4d5e821305500a020dc0]). Expected version 2 got 0
I20251028 02:59:52.219332 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 9c1d8552cc1941bda2e9f02fba56c9f9 (DEFAULT_TABLE table=default.Table_114e1f461bc14e9aaa950f88bc67938e [id=4e8c5072da4e4ec6a616ba4c7fc91cd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.219566 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 6d94d536c1054613b2ae8ffcb08fbaab from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.219604 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c1d8552cc1941bda2e9f02fba56c9f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.219554 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 2d711941006d4417901135b7761a251f (DEFAULT_TABLE table=default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.219784 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d711941006d4417901135b7761a251f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.220016 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 58934e83e36e42e7b913f01b64bd7e70 (DEFAULT_TABLE table=default.Table_053b1ca3666d4641b101c8c10f846980 [id=7eb5bb61f05d4d79b43385b5e91c5b2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.220220 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58934e83e36e42e7b913f01b64bd7e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.220678 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 2d711941006d4417901135b7761a251f (DEFAULT_TABLE table=default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.220814 8545 tablet_bootstrap.cc:654] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.220932 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d711941006d4417901135b7761a251f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.222236 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 99c37c774ae2458989377a8bac94f4e8 (DEFAULT_TABLE table=default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.222429 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c37c774ae2458989377a8bac94f4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.223089 8544 tablet_bootstrap.cc:492] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.223182 8544 ts_tablet_manager.cc:1403] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 02:59:52.220678 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 99c37c774ae2458989377a8bac94f4e8 (DEFAULT_TABLE table=default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.223655 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c37c774ae2458989377a8bac94f4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.223661 8544 raft_consensus.cc:359] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.223768 8544 raft_consensus.cc:385] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.223803 8544 raft_consensus.cc:740] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.223948 8544 consensus_queue.cc:260] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.224260 8544 ts_tablet_manager.cc:1434] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.224468 8544 tablet_bootstrap.cc:492] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.225463 8544 tablet_bootstrap.cc:654] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.226528 8544 tablet_bootstrap.cc:492] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.226603 8544 ts_tablet_manager.cc:1403] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:52.226999 8544 raft_consensus.cc:359] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.227100 8544 raft_consensus.cc:385] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.227133 8544 raft_consensus.cc:740] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.227241 8544 consensus_queue.cc:260] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.227571 8544 ts_tablet_manager.cc:1434] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.227742 8544 tablet_bootstrap.cc:492] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.228240 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 552c509b32374f5f9e3c232fbeddf9bd (DEFAULT_TABLE table=default.Table_e3e2db3778b049ce8d0327aa81f7e2ae [id=83c18d626c264667b358e53e0b6fda63]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.228521 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552c509b32374f5f9e3c232fbeddf9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.228883 8544 tablet_bootstrap.cc:654] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.229306 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 76f4055230904a25a1241a0a86a059d2 (DEFAULT_TABLE table=default.Table_d2e5ac67a4414c03b7890dea514caeee [id=c8f1a159643642eb8dc2f202d73532fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.229542 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76f4055230904a25a1241a0a86a059d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.229694 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 552c509b32374f5f9e3c232fbeddf9bd (DEFAULT_TABLE table=default.Table_e3e2db3778b049ce8d0327aa81f7e2ae [id=83c18d626c264667b358e53e0b6fda63]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.229977 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552c509b32374f5f9e3c232fbeddf9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.231626 8261 tablet_service.cc:1505] Processing CreateTablet for tablet db6eadc5927d4876af9a50b44be97a63 (DEFAULT_TABLE table=default.Table_12396134528e4544a80934aa91d07353 [id=86c2f59df3dd4d2195d18835dceff79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.231841 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db6eadc5927d4876af9a50b44be97a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.232709 8544 tablet_bootstrap.cc:492] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.232805 8544 ts_tablet_manager.cc:1403] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:52.233296 8544 raft_consensus.cc:359] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.233404 8544 raft_consensus.cc:385] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.233436 8544 raft_consensus.cc:740] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.233561 8544 consensus_queue.cc:260] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.233844 8544 ts_tablet_manager.cc:1434] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.234026 8544 tablet_bootstrap.cc:492] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.235164 8544 tablet_bootstrap.cc:654] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.235126 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 58934e83e36e42e7b913f01b64bd7e70 (DEFAULT_TABLE table=default.Table_053b1ca3666d4641b101c8c10f846980 [id=7eb5bb61f05d4d79b43385b5e91c5b2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.235391 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58934e83e36e42e7b913f01b64bd7e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.236225 8544 tablet_bootstrap.cc:492] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.236301 8544 ts_tablet_manager.cc:1403] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20251028 02:59:52.236737 8544 raft_consensus.cc:359] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.236840 8544 raft_consensus.cc:385] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.236882 8544 raft_consensus.cc:740] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.237016 8544 consensus_queue.cc:260] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.237315 8544 ts_tablet_manager.cc:1434] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.237527 8544 tablet_bootstrap.cc:492] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.238442 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 7e347497892b4add9f7b4e19bde67878 (DEFAULT_TABLE table=default.Table_33402cf4a2024ce3b5e0f7471e82322c [id=ccea14281b2c420a99cede953a8de44d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.238688 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e347497892b4add9f7b4e19bde67878. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.239147 8544 tablet_bootstrap.cc:654] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.240617 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 71909cad34b044b093a94397c4b9f29c from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.240911 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 2d711941006d4417901135b7761a251f (DEFAULT_TABLE table=default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.241151 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d711941006d4417901135b7761a251f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.241602 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3adef41531ff4ba0b4f92f0bd13ad3da" } modify_external_catalogs: true
I20251028 02:59:52.241722 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3adef41531ff4ba0b4f92f0bd13ad3da" } modify_external_catalogs: true
I20251028 02:59:52.241694 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 76f4055230904a25a1241a0a86a059d2 (DEFAULT_TABLE table=default.Table_d2e5ac67a4414c03b7890dea514caeee [id=c8f1a159643642eb8dc2f202d73532fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.242013 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 7e347497892b4add9f7b4e19bde67878 (DEFAULT_TABLE table=default.Table_33402cf4a2024ce3b5e0f7471e82322c [id=ccea14281b2c420a99cede953a8de44d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.242238 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e347497892b4add9f7b4e19bde67878. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.242404 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 66fb0173e40e40999f90f015ddb294b7 (table default.Table_eba1e050b6d54c418fbfec1ffcfc8052 [id=d2babb92c2514978b979b30ab6c511f8]) successfully deleted
I20251028 02:59:52.243496 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 99c37c774ae2458989377a8bac94f4e8 (DEFAULT_TABLE table=default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.243706 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c37c774ae2458989377a8bac94f4e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.243852 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 552c509b32374f5f9e3c232fbeddf9bd (DEFAULT_TABLE table=default.Table_e3e2db3778b049ce8d0327aa81f7e2ae [id=83c18d626c264667b358e53e0b6fda63]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.244062 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552c509b32374f5f9e3c232fbeddf9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.244309 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 58934e83e36e42e7b913f01b64bd7e70 (DEFAULT_TABLE table=default.Table_053b1ca3666d4641b101c8c10f846980 [id=7eb5bb61f05d4d79b43385b5e91c5b2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.244493 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58934e83e36e42e7b913f01b64bd7e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.245325 8394 tablet_service.cc:1505] Processing CreateTablet for tablet db6eadc5927d4876af9a50b44be97a63 (DEFAULT_TABLE table=default.Table_12396134528e4544a80934aa91d07353 [id=86c2f59df3dd4d2195d18835dceff79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.245380 8123 tablet_service.cc:1505] Processing CreateTablet for tablet db6eadc5927d4876af9a50b44be97a63 (DEFAULT_TABLE table=default.Table_12396134528e4544a80934aa91d07353 [id=86c2f59df3dd4d2195d18835dceff79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.245587 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db6eadc5927d4876af9a50b44be97a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.246070 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 7e347497892b4add9f7b4e19bde67878 (DEFAULT_TABLE table=default.Table_33402cf4a2024ce3b5e0f7471e82322c [id=ccea14281b2c420a99cede953a8de44d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.246448 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 4132a2b416cc434fbaebfd7df7e5d878 (table default.Table_0424d53c61864c5cbaeb7c12b0b6d96c [id=4d9d3f97f0d541f7a90885b9ab852d83]). Expected version 1 got 0
I20251028 02:59:52.246461 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db6eadc5927d4876af9a50b44be97a63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.246750 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76f4055230904a25a1241a0a86a059d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.248608 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e347497892b4add9f7b4e19bde67878. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.250435 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2be7be7c732e4780859cbe6624edf54e
I20251028 02:59:52.251696 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6d94d536c1054613b2ae8ffcb08fbaab
I20251028 02:59:52.251832 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 2be7be7c732e4780859cbe6624edf54e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.251976 7961 catalog_manager.cc:6719] ReplaceTablet: tablet 6d94d536c1054613b2ae8ffcb08fbaab deleted and replaced by tablet df10bbbd757a4da8905718b61cbb10c8
I20251028 02:59:52.252877 8148 raft_consensus.cc:3060] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:52.252957 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 2be7be7c732e4780859cbe6624edf54e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2be7be7c732e4780859cbe6624edf54e already in progress: creating tablet
I20251028 02:59:52.252996 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d94d536c1054613b2ae8ffcb08fbaab with delete_type TABLET_DATA_DELETED (replaced by tablet df10bbbd757a4da8905718b61cbb10c8) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.253270 8283 raft_consensus.cc:1275] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:52.253656 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d928418826a84f63803ae474dc05de7e
I20251028 02:59:52.253679 8616 consensus_queue.cc:1048] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.253926 7968 catalog_manager.cc:6719] ReplaceTablet: tablet d928418826a84f63803ae474dc05de7e deleted and replaced by tablet da1563d65b4d4ab48325a4f08ee89ed5
I20251028 02:59:52.254190 8148 raft_consensus.cc:1275] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:52.254274 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet d928418826a84f63803ae474dc05de7e with delete_type TABLET_DATA_DELETED (replaced by tablet da1563d65b4d4ab48325a4f08ee89ed5) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.254401 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d94d536c1054613b2ae8ffcb08fbaab with delete_type TABLET_DATA_DELETED (replaced by tablet df10bbbd757a4da8905718b61cbb10c8) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.254565 8551 tablet_replica.cc:333] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.254698 8551 raft_consensus.cc:2243] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:52.254838 8551 pending_rounds.cc:70] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:52.254875 8551 pending_rounds.cc:77] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597126233989120 op_type: NO_OP noop_request { }
I20251028 02:59:52.254930 8551 raft_consensus.cc:2889] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:52.254980 8551 raft_consensus.cc:2272] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.255407 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet d928418826a84f63803ae474dc05de7e with delete_type TABLET_DATA_DELETED (replaced by tablet da1563d65b4d4ab48325a4f08ee89ed5) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.255421 8616 consensus_queue.cc:1048] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.256035 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet d928418826a84f63803ae474dc05de7e with delete_type TABLET_DATA_DELETED (replaced by tablet da1563d65b4d4ab48325a4f08ee89ed5) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:52.256353 8654 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57400: Illegal state: RaftConsensus is not running: State = Stopped
W20251028 02:59:52.257838 7955 catalog_manager.cc:5064] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): alter failed for tablet 6d94d536c1054613b2ae8ffcb08fbaab (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]): Illegal state: RaftConsensus is not running: State = Stopped
I20251028 02:59:52.257853 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 2be7be7c732e4780859cbe6624edf54e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.257896 8545 tablet_bootstrap.cc:492] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.257977 8545 ts_tablet_manager.cc:1403] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20251028 02:59:52.258431 8545 raft_consensus.cc:359] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.258495 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d94d536c1054613b2ae8ffcb08fbaab with delete_type TABLET_DATA_DELETED (replaced by tablet df10bbbd757a4da8905718b61cbb10c8) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.258524 8545 raft_consensus.cc:385] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.258562 8545 raft_consensus.cc:740] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.258687 8545 consensus_queue.cc:260] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.258993 8545 ts_tablet_manager.cc:1434] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.259176 8545 tablet_bootstrap.cc:492] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.259059 7966 catalog_manager.cc:5649] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:52.260877 8545 tablet_bootstrap.cc:654] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.262729 8551 ts_tablet_manager.cc:1916] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:52.263525 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet d928418826a84f63803ae474dc05de7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d928418826a84f63803ae474dc05de7e already in progress: creating tablet
W20251028 02:59:52.263744 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 2be7be7c732e4780859cbe6624edf54e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2be7be7c732e4780859cbe6624edf54e already in progress: creating tablet
W20251028 02:59:52.263870 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6d94d536c1054613b2ae8ffcb08fbaab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d94d536c1054613b2ae8ffcb08fbaab already in progress: creating tablet
I20251028 02:59:52.263932 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 4132a2b416cc434fbaebfd7df7e5d878 (table default.Table_0424d53c61864c5cbaeb7c12b0b6d96c [id=4d9d3f97f0d541f7a90885b9ab852d83]). Expected version 1 got 0
I20251028 02:59:52.264118 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 1c81698dcbe14d5ba61471e7249705db (table default.Table_310089f9c2b34a7e9e1c01c97fdb9b11 [id=c103fff3458d4d08affefb7c38d35e52]). Expected version 1 got 0
I20251028 02:59:52.265190 8551 ts_tablet_manager.cc:1929] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.265264 8551 log.cc:1199] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/6d94d536c1054613b2ae8ffcb08fbaab
I20251028 02:59:52.265445 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e5f0326a03040de81a506caa80dcbd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.265537 8551 ts_tablet_manager.cc:1950] T 6d94d536c1054613b2ae8ffcb08fbaab P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.265631 8550 tablet_replica.cc:333] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.265749 8550 raft_consensus.cc:2243] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.265841 8550 raft_consensus.cc:2272] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.266216 8551 tablet_replica.cc:333] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.266302 8550 ts_tablet_manager.cc:1916] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.266346 8551 raft_consensus.cc:2243] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.266431 8551 raft_consensus.cc:2272] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.266839 8551 ts_tablet_manager.cc:1916] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.267691 8544 tablet_bootstrap.cc:492] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.267776 8544 ts_tablet_manager.cc:1403] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.001s
I20251028 02:59:52.268308 8544 raft_consensus.cc:359] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.268411 8544 raft_consensus.cc:385] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.268448 8544 raft_consensus.cc:740] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.268539 8550 ts_tablet_manager.cc:1929] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.268610 8550 log.cc:1199] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3e5f0326a03040de81a506caa80dcbd9
I20251028 02:59:52.268573 8544 consensus_queue.cc:260] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.268756 8551 ts_tablet_manager.cc:1929] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.268814 8550 ts_tablet_manager.cc:1950] T 3e5f0326a03040de81a506caa80dcbd9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.268831 8551 log.cc:1199] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/d928418826a84f63803ae474dc05de7e
I20251028 02:59:52.268921 8544 ts_tablet_manager.cc:1434] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.269012 8551 ts_tablet_manager.cc:1950] T d928418826a84f63803ae474dc05de7e P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.269896 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 71909cad34b044b093a94397c4b9f29c
I20251028 02:59:52.270061 8545 tablet_bootstrap.cc:492] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.270148 8545 ts_tablet_manager.cc:1403] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:52.270190 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 71909cad34b044b093a94397c4b9f29c deleted and replaced by tablet 37dcf605a2984be1817d9621e4a41448
I20251028 02:59:52.270607 8545 raft_consensus.cc:359] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.270722 8545 raft_consensus.cc:385] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.270776 8545 raft_consensus.cc:740] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.270911 8545 consensus_queue.cc:260] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.271210 8545 ts_tablet_manager.cc:1434] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.271441 8545 tablet_bootstrap.cc:492] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.272331 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 4132a2b416cc434fbaebfd7df7e5d878 (table default.Table_0424d53c61864c5cbaeb7c12b0b6d96c [id=4d9d3f97f0d541f7a90885b9ab852d83]). Expected version 1 got 0
I20251028 02:59:52.272472 8545 tablet_bootstrap.cc:654] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.273324 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 71909cad34b044b093a94397c4b9f29c with delete_type TABLET_DATA_DELETED (replaced by tablet 37dcf605a2984be1817d9621e4a41448) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.273501 8550 tablet_replica.cc:333] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.273672 8550 raft_consensus.cc:2243] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.273803 8550 raft_consensus.cc:2272] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.273937 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3e5f0326a03040de81a506caa80dcbd9 (table default.Table_c320b509540344b7895fb967b844a408 [id=fbecd1cf849b47bb880b96a56333fbd9]) successfully deleted
I20251028 02:59:52.274899 8544 tablet_bootstrap.cc:492] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.276424 8550 ts_tablet_manager.cc:1916] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.277084 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906
I20251028 02:59:52.277185 8912 raft_consensus.cc:493] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.277293 8912 raft_consensus.cc:515] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.277776 8545 tablet_bootstrap.cc:492] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.277783 8912 leader_election.cc:290] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.277889 8545 ts_tablet_manager.cc:1403] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20251028 02:59:52.278235 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c24384fecd7b4c0a9b2e3da1e0369866" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.278446 8545 raft_consensus.cc:359] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.278568 8545 raft_consensus.cc:385] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.278613 8545 raft_consensus.cc:740] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.278765 8545 consensus_queue.cc:260] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.278808 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c24384fecd7b4c0a9b2e3da1e0369866" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.278957 8421 raft_consensus.cc:2393] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:52.279290 7963 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 4132a2b416cc434fbaebfd7df7e5d878 (table default.Table_0424d53c61864c5cbaeb7c12b0b6d96c [id=4d9d3f97f0d541f7a90885b9ab852d83]). Expected version 1 got 0
I20251028 02:59:52.279286 8083 leader_election.cc:304] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:52.279575 8912 raft_consensus.cc:3060] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.279654 8550 ts_tablet_manager.cc:1929] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.279742 8550 log.cc:1199] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/71909cad34b044b093a94397c4b9f29c
I20251028 02:59:52.280019 8550 ts_tablet_manager.cc:1950] T 71909cad34b044b093a94397c4b9f29c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.280532 8912 raft_consensus.cc:2749] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:52.281114 8545 ts_tablet_manager.cc:1434] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:52.281313 8545 tablet_bootstrap.cc:492] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.283401 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.283855 8545 tablet_bootstrap.cc:654] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.284719 7969 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 71feaa313a8f45798561741f7b5d9b20 (table default.Table_5233b8b0ea95426a8497f5cbd89716dc [id=5581f43ea3854dbeba821353ffce6ec7]). Expected version 1 got 0
I20251028 02:59:52.285166 8545 tablet_bootstrap.cc:492] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.285262 8545 ts_tablet_manager.cc:1403] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:52.285858 8545 raft_consensus.cc:359] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.285969 8545 raft_consensus.cc:385] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.286002 8545 raft_consensus.cc:740] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.286131 8545 consensus_queue.cc:260] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.287190 8912 raft_consensus.cc:493] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.287349 8912 raft_consensus.cc:515] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.287763 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 2be7be7c732e4780859cbe6624edf54e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.288036 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63ee5aa5a6e841fdb9244563d75394ac" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.288172 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 71909cad34b044b093a94397c4b9f29c with delete_type TABLET_DATA_DELETED (replaced by tablet 37dcf605a2984be1817d9621e4a41448) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.288313 8551 tablet_replica.cc:333] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.288435 8551 raft_consensus.cc:2243] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.288538 8551 raft_consensus.cc:2272] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.288547 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 71909cad34b044b093a94397c4b9f29c (table default.Table_d012c9d921f542c4ab7a5e8858d351d4 [id=739f0673774d472bbb62f176b70998bc]) successfully deleted
I20251028 02:59:52.289077 8551 ts_tablet_manager.cc:1916] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.289212 8545 ts_tablet_manager.cc:1434] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:52.289434 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.289570 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63ee5aa5a6e841fdb9244563d75394ac" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.289727 8285 raft_consensus.cc:2393] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 02:59:52.289834 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet a770dec97abb478b8c5504cc007b6092 with delete_type TABLET_DATA_DELETED (replaced by tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.290061 8083 leader_election.cc:304] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:52.290133 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 909b08e0860b4e4f9bb1822acfd1dc02 with delete_type TABLET_DATA_DELETED (replaced by tablet 8b88fda3f40540f3b42ee7f33dd33251) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.290246 8562 raft_consensus.cc:3060] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.291224 8562 raft_consensus.cc:2749] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:52.291538 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 71909cad34b044b093a94397c4b9f29c with delete_type TABLET_DATA_DELETED (replaced by tablet 37dcf605a2984be1817d9621e4a41448) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.291807 8912 leader_election.cc:290] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.292121 8551 ts_tablet_manager.cc:1929] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.292197 8551 log.cc:1199] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/71909cad34b044b093a94397c4b9f29c
I20251028 02:59:52.292433 8551 ts_tablet_manager.cc:1950] T 71909cad34b044b093a94397c4b9f29c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.293012 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.293131 8551 tablet_replica.cc:333] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.293267 8551 raft_consensus.cc:2243] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.293385 8551 raft_consensus.cc:2272] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.290562 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1692c48a1cb243039125140ab7932f6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:52.293491 7969 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 8cb97d5192684d748b8b485f891b0dc0 (table default.Table_35fd4c0e400244f28d1a855779611c46 [id=a9662d8045574f0abf81c9e6bc2f180c]). Expected version 1 got 0
W20251028 02:59:52.293622 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 already in progress: creating tablet
I20251028 02:59:52.293694 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e69b7cea080549b1b8a67cb91703a472"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.293887 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1692c48a1cb243039125140ab7932f6b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:52.293922 7969 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet 8cb97d5192684d748b8b485f891b0dc0 (table default.Table_35fd4c0e400244f28d1a855779611c46 [id=a9662d8045574f0abf81c9e6bc2f180c], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 8cb97d5192684d748b8b485f891b0dc0 consensus state has no leader
W20251028 02:59:52.293942 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e69b7cea080549b1b8a67cb91703a472 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.291018 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_dddd2e3cad6f4d96bd57539e6195677a" } modify_external_catalogs: true
I20251028 02:59:52.291000 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b55d0b45f7e543e8a5927bde50d0502b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.294385 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b55d0b45f7e543e8a5927bde50d0502b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.291246 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet 3e7dbafe267a4c62b0418d1b26585aad from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.291326 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_35fd4c0e400244f28d1a855779611c46" } new_table_name: "default.Table_8bd9c3af53644cbcb68461ae9433a9f1" modify_external_catalogs: true
I20251028 02:59:52.291399 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_114e1f461bc14e9aaa950f88bc67938e" } new_table_name: "default.Table_bb551b322c5140ba89d154ff450085dd" modify_external_catalogs: true
I20251028 02:59:52.297719 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_dddd2e3cad6f4d96bd57539e6195677a" } modify_external_catalogs: true
I20251028 02:59:52.291427 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 6d94d536c1054613b2ae8ffcb08fbaab (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]) successfully deleted
I20251028 02:59:52.298923 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet d928418826a84f63803ae474dc05de7e (table default.Table_27a9af4382b34a3eab870c4d40a6c5bf [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
I20251028 02:59:52.299014 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 71909cad34b044b093a94397c4b9f29c (table default.Table_d012c9d921f542c4ab7a5e8858d351d4 [id=739f0673774d472bbb62f176b70998bc]) successfully deleted
W20251028 02:59:52.299103 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 already in progress: creating tablet
I20251028 02:59:52.291517 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_17aa90b035264d2585988c8243238e7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.299621 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_17aa90b035264d2585988c8243238e7a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.299858 8551 ts_tablet_manager.cc:1916] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.277191 8544 tablet_bootstrap.cc:654] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.291674 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bcdd1e4fff1a45e39b03d0c7f2268672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.300540 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcdd1e4fff1a45e39b03d0c7f2268672 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.302129 8551 ts_tablet_manager.cc:1929] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.302206 8551 log.cc:1199] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a770dec97abb478b8c5504cc007b6092
I20251028 02:59:52.302405 8551 ts_tablet_manager.cc:1950] T a770dec97abb478b8c5504cc007b6092 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.302842 8552 ts_tablet_manager.cc:1929] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.303052 8551 tablet_replica.cc:333] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.303195 8551 raft_consensus.cc:2243] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.303189 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a770dec97abb478b8c5504cc007b6092 (table default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]) successfully deleted
I20251028 02:59:52.304782 8544 tablet_bootstrap.cc:492] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.305243 8544 ts_tablet_manager.cc:1403] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.001s
I20251028 02:59:52.305693 8544 raft_consensus.cc:359] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.305830 8544 raft_consensus.cc:385] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.305876 8544 raft_consensus.cc:740] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.306027 8544 consensus_queue.cc:260] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.306425 8545 tablet_bootstrap.cc:492] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.307566 8545 tablet_bootstrap.cc:654] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.308012 8544 ts_tablet_manager.cc:1434] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:52.308264 8656 consensus_queue.cc:1048] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:52.308311 7955 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 71feaa313a8f45798561741f7b5d9b20 (table default.Table_5233b8b0ea95426a8497f5cbd89716dc [id=5581f43ea3854dbeba821353ffce6ec7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 71feaa313a8f45798561741f7b5d9b20 consensus state has no leader
I20251028 02:59:52.308404 8544 tablet_bootstrap.cc:492] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.309314 8544 tablet_bootstrap.cc:654] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.310542 8753 tablet.cc:1722] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:52.312520 8551 raft_consensus.cc:2272] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.314157 8551 ts_tablet_manager.cc:1916] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.316380 8551 ts_tablet_manager.cc:1929] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.316459 8551 log.cc:1199] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/909b08e0860b4e4f9bb1822acfd1dc02
I20251028 02:59:52.316668 8551 ts_tablet_manager.cc:1950] T 909b08e0860b4e4f9bb1822acfd1dc02 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.317410 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 909b08e0860b4e4f9bb1822acfd1dc02 (table default.Table_7fb744c8c91e4c9daadacdab1a408fa1 [id=98a07edd5f664674a2d33344038a8801]) successfully deleted
I20251028 02:59:52.317624 8545 tablet_bootstrap.cc:492] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.317723 8545 ts_tablet_manager.cc:1403] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:52.318212 8545 raft_consensus.cc:359] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.318322 8545 raft_consensus.cc:385] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.318360 8545 raft_consensus.cc:740] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.318472 8545 consensus_queue.cc:260] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.318859 8545 ts_tablet_manager.cc:1434] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.302953 8552 log.cc:1199] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/f4bebc4575e4462cbc8a02fca4fb228b
I20251028 02:59:52.319347 8545 tablet_bootstrap.cc:492] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.318291 8884 tablet.cc:1722] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:52.320727 8545 tablet_bootstrap.cc:654] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.321595 8420 raft_consensus.cc:1275] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:52.321627 8146 raft_consensus.cc:3060] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.321691 8552 ts_tablet_manager.cc:1950] T f4bebc4575e4462cbc8a02fca4fb228b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.322016 8553 consensus_queue.cc:1048] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.322717 8146 raft_consensus.cc:1275] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:52.323235 8545 tablet_bootstrap.cc:492] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.323336 8545 ts_tablet_manager.cc:1403] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:52.323343 8544 tablet_bootstrap.cc:492] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.323423 8544 ts_tablet_manager.cc:1403] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20251028 02:59:52.323654 8546 tablet_bootstrap.cc:492] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.323734 8546 ts_tablet_manager.cc:1403] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.171s user 0.001s sys 0.001s
I20251028 02:59:52.323810 8545 raft_consensus.cc:359] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.323907 8545 raft_consensus.cc:385] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.323894 8544 raft_consensus.cc:359] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.323952 8545 raft_consensus.cc:740] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.323989 8544 raft_consensus.cc:385] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.324015 8544 raft_consensus.cc:740] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.324067 8545 consensus_queue.cc:260] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.324178 8544 consensus_queue.cc:260] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.324265 8546 raft_consensus.cc:359] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.324357 8553 consensus_queue.cc:1048] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.325125 8545 ts_tablet_manager.cc:1434] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:52.325314 8545 tablet_bootstrap.cc:492] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.325598 8544 ts_tablet_manager.cc:1434] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:52.325771 8544 tablet_bootstrap.cc:492] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.326385 8545 tablet_bootstrap.cc:654] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.327430 8544 tablet_bootstrap.cc:654] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.328576 8649 mvcc.cc:204] Tried to move back new op lower bound from 7215597126944821248 to 7215597126300033024. Current Snapshot: MvccSnapshot[applied={T|T < 7215597126944821248}]
I20251028 02:59:52.324368 8546 raft_consensus.cc:385] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.328725 8546 raft_consensus.cc:740] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.328851 8546 consensus_queue.cc:260] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.329156 8546 ts_tablet_manager.cc:1434] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 02:59:52.329334 8546 tablet_bootstrap.cc:492] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.329888 8726 tablet.cc:1722] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:52.329905 8884 tablet.cc:1722] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:52.330528 8546 tablet_bootstrap.cc:654] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.330789 8753 tablet.cc:1722] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:52.331331 8654 mvcc.cc:204] Tried to move back new op lower bound from 7215597126944821248 to 7215597126300033024. Current Snapshot: MvccSnapshot[applied={T|T < 7215597126944821248}]
I20251028 02:59:52.331487 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet f4bebc4575e4462cbc8a02fca4fb228b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.331983 8550 tablet_replica.cc:333] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.332108 8550 raft_consensus.cc:2243] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.332194 8550 raft_consensus.cc:2272] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.334442 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3e7dbafe267a4c62b0418d1b26585aad
I20251028 02:59:52.334800 7962 catalog_manager.cc:6719] ReplaceTablet: tablet 3e7dbafe267a4c62b0418d1b26585aad deleted and replaced by tablet 68ca4ff164b24b7498f953ec11cf46be
I20251028 02:59:52.335016 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_587714a762d0451d98e0b71b42c3834c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:52.335263 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c24384fecd7b4c0a9b2e3da1e0369866
W20251028 02:59:52.335474 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_587714a762d0451d98e0b71b42c3834c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.335670 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b4b24abe267c451f95fc63bf0234d06c" } modify_external_catalogs: true
I20251028 02:59:52.335712 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e7dbafe267a4c62b0418d1b26585aad with delete_type TABLET_DATA_DELETED (replaced by tablet 68ca4ff164b24b7498f953ec11cf46be) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.335750 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b4b24abe267c451f95fc63bf0234d06c" } modify_external_catalogs: true
I20251028 02:59:52.335927 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e7dbafe267a4c62b0418d1b26585aad with delete_type TABLET_DATA_DELETED (replaced by tablet 68ca4ff164b24b7498f953ec11cf46be) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.335028 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e3e2db3778b049ce8d0327aa81f7e2ae" } modify_external_catalogs: true
I20251028 02:59:52.336028 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e3e2db3778b049ce8d0327aa81f7e2ae" } modify_external_catalogs: true
W20251028 02:59:52.336313 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8cb97d5192684d748b8b485f891b0dc0 consensus state has no leader
I20251028 02:59:52.336421 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet c24384fecd7b4c0a9b2e3da1e0369866 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.336520 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet c24384fecd7b4c0a9b2e3da1e0369866 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.337210 8551 tablet_replica.cc:333] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.337347 8551 raft_consensus.cc:2243] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:52.337649 8551 raft_consensus.cc:2272] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.338053 8550 ts_tablet_manager.cc:1916] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.334779 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c3d6504130c54bdeade5bb23b6cd0679"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.339753 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c3d6504130c54bdeade5bb23b6cd0679 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:52.339910 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9c1d8552cc1941bda2e9f02fba56c9f9 consensus state has no leader
I20251028 02:59:52.340190 8550 ts_tablet_manager.cc:1929] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.340260 8550 log.cc:1199] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/f4bebc4575e4462cbc8a02fca4fb228b
I20251028 02:59:52.340467 8550 ts_tablet_manager.cc:1950] T f4bebc4575e4462cbc8a02fca4fb228b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.340566 8551 ts_tablet_manager.cc:1916] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.342877 8551 ts_tablet_manager.cc:1929] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:52.342952 8551 log.cc:1199] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3e7dbafe267a4c62b0418d1b26585aad
I20251028 02:59:52.343212 8551 ts_tablet_manager.cc:1950] T 3e7dbafe267a4c62b0418d1b26585aad P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.345479 8546 tablet_bootstrap.cc:492] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.345579 8546 ts_tablet_manager.cc:1403] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 02:59:52.346108 8546 raft_consensus.cc:359] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.346225 8546 raft_consensus.cc:385] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.346259 8546 raft_consensus.cc:740] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.346386 8546 consensus_queue.cc:260] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.346657 8546 ts_tablet_manager.cc:1434] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.346881 8546 tablet_bootstrap.cc:492] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.348140 8544 tablet_bootstrap.cc:492] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.348198 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3e7dbafe267a4c62b0418d1b26585aad (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:52.348239 8544 ts_tablet_manager.cc:1403] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.001s
I20251028 02:59:52.348433 8551 tablet_replica.cc:333] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.348591 8551 raft_consensus.cc:2243] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.348685 8551 raft_consensus.cc:2272] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.348729 8544 raft_consensus.cc:359] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.348830 8544 raft_consensus.cc:385] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.348860 8544 raft_consensus.cc:740] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.348971 8544 consensus_queue.cc:260] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.349269 8544 ts_tablet_manager.cc:1434] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:52.349460 8544 tablet_bootstrap.cc:492] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.350319 8546 tablet_bootstrap.cc:654] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.350512 8544 tablet_bootstrap.cc:654] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.350772 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet c24384fecd7b4c0a9b2e3da1e0369866 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.350930 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e7dbafe267a4c62b0418d1b26585aad with delete_type TABLET_DATA_DELETED (replaced by tablet 68ca4ff164b24b7498f953ec11cf46be) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.352097 8551 ts_tablet_manager.cc:1916] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.353847 8546 tablet_bootstrap.cc:492] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.353941 8546 ts_tablet_manager.cc:1403] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20251028 02:59:52.354439 8546 raft_consensus.cc:359] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.354544 8546 raft_consensus.cc:385] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.354586 8546 raft_consensus.cc:740] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.354465 8551 ts_tablet_manager.cc:1929] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.354710 8551 log.cc:1199] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c24384fecd7b4c0a9b2e3da1e0369866
I20251028 02:59:52.354712 8546 consensus_queue.cc:260] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.352968 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet f4bebc4575e4462cbc8a02fca4fb228b (table default.Table_deda042aff7640a3afd2f724eba68912 [id=e7c4b52cbf8c496e94ca8634d53e41de]) successfully deleted
I20251028 02:59:52.354945 8551 ts_tablet_manager.cc:1950] T c24384fecd7b4c0a9b2e3da1e0369866 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.354964 8550 tablet_replica.cc:333] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.354997 8546 ts_tablet_manager.cc:1434] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.355105 8550 raft_consensus.cc:2243] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.355202 8550 raft_consensus.cc:2272] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.355202 8546 tablet_bootstrap.cc:492] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.355798 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet c24384fecd7b4c0a9b2e3da1e0369866 (table default.Table_dddd2e3cad6f4d96bd57539e6195677a [id=b82f549c8f03449eaa1762408bc67772]) successfully deleted
I20251028 02:59:52.355877 8550 ts_tablet_manager.cc:1916] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.356042 8544 tablet_bootstrap.cc:492] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.356127 8544 ts_tablet_manager.cc:1403] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20251028 02:59:52.356261 8546 tablet_bootstrap.cc:654] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.356555 8544 raft_consensus.cc:359] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.356650 8544 raft_consensus.cc:385] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.356679 8544 raft_consensus.cc:740] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.356807 8544 consensus_queue.cc:260] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.357103 8544 ts_tablet_manager.cc:1434] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.357281 8544 tablet_bootstrap.cc:492] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.357311 8556 raft_consensus.cc:493] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.357380 8556 raft_consensus.cc:515] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.357730 8556 leader_election.cc:290] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.358083 8550 ts_tablet_manager.cc:1929] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.358141 8550 log.cc:1199] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c24384fecd7b4c0a9b2e3da1e0369866
I20251028 02:59:52.358243 8544 tablet_bootstrap.cc:654] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.358346 8550 ts_tablet_manager.cc:1950] T c24384fecd7b4c0a9b2e3da1e0369866 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.359086 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e20b1f36d3f4f70a93ad8a0b29d6687" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.359195 8146 raft_consensus.cc:2468] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:52.359505 8218 leader_election.cc:304] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:52.359656 8556 raft_consensus.cc:2804] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.359701 8556 raft_consensus.cc:493] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.359740 8556 raft_consensus.cc:3060] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.359898 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet f4bebc4575e4462cbc8a02fca4fb228b (table default.Table_deda042aff7640a3afd2f724eba68912 [id=e7c4b52cbf8c496e94ca8634d53e41de]) successfully deleted
I20251028 02:59:52.359902 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e20b1f36d3f4f70a93ad8a0b29d6687" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.360028 8420 raft_consensus.cc:2468] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:52.360731 8556 raft_consensus.cc:515] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.361088 8556 leader_election.cc:290] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.361652 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e20b1f36d3f4f70a93ad8a0b29d6687" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.361773 8420 raft_consensus.cc:3060] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.362815 8420 raft_consensus.cc:2468] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:52.363205 8550 tablet_replica.cc:333] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.363332 8220 leader_election.cc:304] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.363360 8550 raft_consensus.cc:2243] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.363679 8550 raft_consensus.cc:2272] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.364176 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e20b1f36d3f4f70a93ad8a0b29d6687" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.364282 8146 raft_consensus.cc:3060] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.365002 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 8cb97d5192684d748b8b485f891b0dc0 (table default.Table_8bd9c3af53644cbcb68461ae9433a9f1 [id=a9662d8045574f0abf81c9e6bc2f180c]). Expected version 2 got 0
I20251028 02:59:52.365262 8146 raft_consensus.cc:2468] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:52.365645 8556 raft_consensus.cc:2804] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.365728 8556 raft_consensus.cc:697] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:52.365849 8556 consensus_queue.cc:237] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.366114 8283 raft_consensus.cc:1275] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:52.366675 8616 consensus_queue.cc:1048] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.367686 8146 raft_consensus.cc:1275] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:52.367980 7965 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 (table default.Table_7778bd93ae56485f921a20d0df78a3ec [id=7a2a6ab3c9744942a9413c6cb057f527]). Expected version 4 got 0
I20251028 02:59:52.368116 7965 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 1c81698dcbe14d5ba61471e7249705db (table default.Table_310089f9c2b34a7e9e1c01c97fdb9b11 [id=c103fff3458d4d08affefb7c38d35e52]). Expected version 1 got 0
I20251028 02:59:52.368135 8648 consensus_queue.cc:1048] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.368531 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet a770dec97abb478b8c5504cc007b6092 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.368920 8550 ts_tablet_manager.cc:1916] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.369248 8399 tablet_service.cc:1505] Processing CreateTablet for tablet aeddf70e5bd342f19f3f427d1df73814 (DEFAULT_TABLE table=default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.369495 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeddf70e5bd342f19f3f427d1df73814. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.369606 8262 tablet_service.cc:1505] Processing CreateTablet for tablet aeddf70e5bd342f19f3f427d1df73814 (DEFAULT_TABLE table=default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.369901 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeddf70e5bd342f19f3f427d1df73814. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.371167 8546 tablet_bootstrap.cc:492] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.371253 8546 ts_tablet_manager.cc:1403] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 02:59:52.371485 8753 tablet.cc:1722] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.371807 8546 raft_consensus.cc:359] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.371867 8394 tablet_service.cc:1505] Processing CreateTablet for tablet c048520fe7b248e586e44a2047ffb239 (DEFAULT_TABLE table=default.Table_af395b1589f74f4ebe9dd06515819408 [id=f2d1a37b2cac4447b19d61632e988672]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.371908 8546 raft_consensus.cc:385] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.371948 8546 raft_consensus.cc:740] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.372112 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c048520fe7b248e586e44a2047ffb239. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.372099 8546 consensus_queue.cc:260] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.372377 8546 ts_tablet_manager.cc:1434] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.372546 8546 tablet_bootstrap.cc:492] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.373662 8546 tablet_bootstrap.cc:654] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.374995 8263 tablet_service.cc:1505] Processing CreateTablet for tablet c048520fe7b248e586e44a2047ffb239 (DEFAULT_TABLE table=default.Table_af395b1589f74f4ebe9dd06515819408 [id=f2d1a37b2cac4447b19d61632e988672]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.375238 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c048520fe7b248e586e44a2047ffb239. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.376727 8726 tablet.cc:1722] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20251028 02:59:52.377961 7955 catalog_manager.cc:5058] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): alter failed for tablet 6d94d536c1054613b2ae8ffcb08fbaab (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]),no further retry: Not found: Tablet not found: 6d94d536c1054613b2ae8ffcb08fbaab
I20251028 02:59:52.378863 8260 tablet_service.cc:1505] Processing CreateTablet for tablet da1563d65b4d4ab48325a4f08ee89ed5 (DEFAULT_TABLE table=default.Table_27a9af4382b34a3eab870c4d40a6c5bf [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.379012 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 37dcf605a2984be1817d9621e4a41448 (DEFAULT_TABLE table=default.Table_d012c9d921f542c4ab7a5e8858d351d4 [id=739f0673774d472bbb62f176b70998bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.379125 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da1563d65b4d4ab48325a4f08ee89ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.379261 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37dcf605a2984be1817d9621e4a41448. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.379906 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 50a0a3d690154de7acd55f37ab736d13 (DEFAULT_TABLE table=default.Table_b9407dac1631402082afb2b12dc61dbd [id=a015c31e18c74191879fe1474dcb6ab2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.381624 8400 tablet_service.cc:1505] Processing CreateTablet for tablet df10bbbd757a4da8905718b61cbb10c8 (DEFAULT_TABLE table=default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.379012 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 50a0a3d690154de7acd55f37ab736d13 (DEFAULT_TABLE table=default.Table_b9407dac1631402082afb2b12dc61dbd [id=a015c31e18c74191879fe1474dcb6ab2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.382020 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50a0a3d690154de7acd55f37ab736d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.383797 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5233b8b0ea95426a8497f5cbd89716dc" } new_table_name: "default.Table_d5e26eda7deb4e4f9d3a93ac052b01c9" modify_external_catalogs: true
I20251028 02:59:52.385169 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50a0a3d690154de7acd55f37ab736d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.385365 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df10bbbd757a4da8905718b61cbb10c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.385548 8263 tablet_service.cc:1505] Processing CreateTablet for tablet df10bbbd757a4da8905718b61cbb10c8 (DEFAULT_TABLE table=default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.385802 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df10bbbd757a4da8905718b61cbb10c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.386257 8552 tablet_replica.cc:333] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.386400 8552 raft_consensus.cc:2243] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.386513 8552 raft_consensus.cc:2272] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.387591 8552 ts_tablet_manager.cc:1916] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.388000 8399 tablet_service.cc:1505] Processing CreateTablet for tablet da1563d65b4d4ab48325a4f08ee89ed5 (DEFAULT_TABLE table=default.Table_27a9af4382b34a3eab870c4d40a6c5bf [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.388247 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da1563d65b4d4ab48325a4f08ee89ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.389778 8552 ts_tablet_manager.cc:1929] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.389850 8552 log.cc:1199] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/32ed23fd3cdc4be9ab0e65633c86e787
I20251028 02:59:52.390072 8552 ts_tablet_manager.cc:1950] T 32ed23fd3cdc4be9ab0e65633c86e787 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.390662 8552 tablet_replica.cc:333] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.390805 8552 raft_consensus.cc:2243] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.390908 8552 raft_consensus.cc:2272] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.379906 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 37dcf605a2984be1817d9621e4a41448 (DEFAULT_TABLE table=default.Table_d012c9d921f542c4ab7a5e8858d351d4 [id=739f0673774d472bbb62f176b70998bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.391649 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37dcf605a2984be1817d9621e4a41448. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.401521 8545 tablet_bootstrap.cc:492] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.401636 8545 ts_tablet_manager.cc:1403] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.076s user 0.002s sys 0.000s
I20251028 02:59:52.402114 8545 raft_consensus.cc:359] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.402217 8545 raft_consensus.cc:385] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.402246 8545 raft_consensus.cc:740] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.402360 8545 consensus_queue.cc:260] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.402606 8545 ts_tablet_manager.cc:1434] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.402786 8545 tablet_bootstrap.cc:492] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.403250 8726 tablet.cc:1722] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:52.403710 8552 ts_tablet_manager.cc:1916] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.403928 8545 tablet_bootstrap.cc:654] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.405773 8550 ts_tablet_manager.cc:1929] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:52.405884 8550 log.cc:1199] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3e7dbafe267a4c62b0418d1b26585aad
I20251028 02:59:52.406143 8550 ts_tablet_manager.cc:1950] T 3e7dbafe267a4c62b0418d1b26585aad P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.406458 8552 ts_tablet_manager.cc:1929] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.406605 8552 log.cc:1199] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/6d94d536c1054613b2ae8ffcb08fbaab
I20251028 02:59:52.406869 8552 ts_tablet_manager.cc:1950] T 6d94d536c1054613b2ae8ffcb08fbaab P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.406896 8122 tablet_service.cc:1505] Processing CreateTablet for tablet aeddf70e5bd342f19f3f427d1df73814 (DEFAULT_TABLE table=default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.407446 8123 tablet_service.cc:1505] Processing CreateTablet for tablet c048520fe7b248e586e44a2047ffb239 (DEFAULT_TABLE table=default.Table_af395b1589f74f4ebe9dd06515819408 [id=f2d1a37b2cac4447b19d61632e988672]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.407786 8552 tablet_replica.cc:333] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.407985 8552 raft_consensus.cc:2243] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.408154 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeddf70e5bd342f19f3f427d1df73814. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.408149 8552 raft_consensus.cc:2272] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.410166 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 552c509b32374f5f9e3c232fbeddf9bd
I20251028 02:59:52.412145 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 6d94d536c1054613b2ae8ffcb08fbaab (table default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]) successfully deleted
I20251028 02:59:52.412168 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 552c509b32374f5f9e3c232fbeddf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.414769 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c048520fe7b248e586e44a2047ffb239. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.416947 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2d711941006d4417901135b7761a251f
I20251028 02:59:52.417210 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 552c509b32374f5f9e3c232fbeddf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.417596 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d711941006d4417901135b7761a251f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:52.418080 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 552c509b32374f5f9e3c232fbeddf9bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 552c509b32374f5f9e3c232fbeddf9bd already in progress: creating tablet
I20251028 02:59:52.418399 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6997dbe5002f4b0594291518fccd3c96" } new_table_name: "default.Table_6b3cfd5f43ca4f3b82a37dab711251b0" modify_external_catalogs: true
I20251028 02:59:52.420562 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d711941006d4417901135b7761a251f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:52.421566 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 2d711941006d4417901135b7761a251f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d711941006d4417901135b7761a251f already in progress: creating tablet
I20251028 02:59:52.423058 8552 ts_tablet_manager.cc:1916] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:52.423961 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 71feaa313a8f45798561741f7b5d9b20 consensus state has no leader
I20251028 02:59:52.424258 7968 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_0424d53c61864c5cbaeb7c12b0b6d96c [id=4d9d3f97f0d541f7a90885b9ab852d83] alter complete (version 1)
I20251028 02:59:52.425741 8552 ts_tablet_manager.cc:1929] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.425814 8552 log.cc:1199] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/d928418826a84f63803ae474dc05de7e
I20251028 02:59:52.426026 8552 ts_tablet_manager.cc:1950] T d928418826a84f63803ae474dc05de7e P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.423375 8123 tablet_service.cc:1505] Processing CreateTablet for tablet df10bbbd757a4da8905718b61cbb10c8 (DEFAULT_TABLE table=default.Table_6997dbe5002f4b0594291518fccd3c96 [id=a6ae2be5375e4328848f28f43dfc49ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.427803 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df10bbbd757a4da8905718b61cbb10c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.428324 8552 tablet_replica.cc:333] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.428550 8552 raft_consensus.cc:2243] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:52.428718 8552 pending_rounds.cc:70] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:52.428752 8552 pending_rounds.cc:77] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597125517504512 op_type: NO_OP noop_request { }
I20251028 02:59:52.428840 8552 raft_consensus.cc:2889] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:52.428910 8552 raft_consensus.cc:2272] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.429431 8552 ts_tablet_manager.cc:1916] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.430219 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet d928418826a84f63803ae474dc05de7e (table default.Table_27a9af4382b34a3eab870c4d40a6c5bf [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
W20251028 02:59:52.430399 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 2be7be7c732e4780859cbe6624edf54e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2be7be7c732e4780859cbe6624edf54e already in progress: creating tablet
I20251028 02:59:52.430963 8123 tablet_service.cc:1505] Processing CreateTablet for tablet da1563d65b4d4ab48325a4f08ee89ed5 (DEFAULT_TABLE table=default.Table_27a9af4382b34a3eab870c4d40a6c5bf [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.431057 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 50a0a3d690154de7acd55f37ab736d13 (DEFAULT_TABLE table=default.Table_b9407dac1631402082afb2b12dc61dbd [id=a015c31e18c74191879fe1474dcb6ab2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.431387 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50a0a3d690154de7acd55f37ab736d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.432592 8552 ts_tablet_manager.cc:1929] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.432662 8552 log.cc:1199] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/71909cad34b044b093a94397c4b9f29c
I20251028 02:59:52.432909 8552 ts_tablet_manager.cc:1950] T 71909cad34b044b093a94397c4b9f29c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.433661 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 71909cad34b044b093a94397c4b9f29c (table default.Table_d012c9d921f542c4ab7a5e8858d351d4 [id=739f0673774d472bbb62f176b70998bc]) successfully deleted
I20251028 02:59:52.433915 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da1563d65b4d4ab48325a4f08ee89ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.433940 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 37dcf605a2984be1817d9621e4a41448 (DEFAULT_TABLE table=default.Table_d012c9d921f542c4ab7a5e8858d351d4 [id=739f0673774d472bbb62f176b70998bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.434231 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37dcf605a2984be1817d9621e4a41448. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:52.435714 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 already in progress: creating tablet
I20251028 02:59:52.436491 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet c24384fecd7b4c0a9b2e3da1e0369866 (table default.Table_dddd2e3cad6f4d96bd57539e6195677a [id=b82f549c8f03449eaa1762408bc67772]) successfully deleted
I20251028 02:59:52.436920 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3e7dbafe267a4c62b0418d1b26585aad (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:52.437359 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 552c509b32374f5f9e3c232fbeddf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.437572 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d711941006d4417901135b7761a251f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.439549 8884 tablet.cc:1722] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.443492 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_50c26dd07fa04d9a838968de2a92f0e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.443974 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50c26dd07fa04d9a838968de2a92f0e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:52.444571 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 552c509b32374f5f9e3c232fbeddf9bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 552c509b32374f5f9e3c232fbeddf9bd already in progress: creating tablet
W20251028 02:59:52.444761 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 2d711941006d4417901135b7761a251f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d711941006d4417901135b7761a251f already in progress: creating tablet
I20251028 02:59:52.445101 8546 tablet_bootstrap.cc:492] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.445154 8544 tablet_bootstrap.cc:492] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.445173 8546 ts_tablet_manager.cc:1403] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.073s user 0.002s sys 0.000s
I20251028 02:59:52.445228 8544 ts_tablet_manager.cc:1403] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.088s user 0.001s sys 0.000s
I20251028 02:59:52.445569 8546 raft_consensus.cc:359] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.445592 8544 raft_consensus.cc:359] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.445647 8546 raft_consensus.cc:385] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.445669 8544 raft_consensus.cc:385] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.445668 8546 raft_consensus.cc:740] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.445691 8544 raft_consensus.cc:740] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.445780 8546 consensus_queue.cc:260] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.445798 8544 consensus_queue.cc:260] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.446048 8544 ts_tablet_manager.cc:1434] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.446058 8546 ts_tablet_manager.cc:1434] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.446231 8544 tablet_bootstrap.cc:492] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.446240 8546 tablet_bootstrap.cc:492] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.447244 8544 tablet_bootstrap.cc:654] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.443466 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_cbf114f7a2674dfa956f0f5d163c7622"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.448016 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cbf114f7a2674dfa956f0f5d163c7622 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:52.449034 8220 consensus_peers.cc:597] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c24384fecd7b4c0a9b2e3da1e0369866. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:52.449147 8546 tablet_bootstrap.cc:654] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.450042 8544 tablet_bootstrap.cc:492] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.450136 8544 ts_tablet_manager.cc:1403] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:52.450091 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e95e72f2a7164cdd902dd9666361348c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.450448 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e95e72f2a7164cdd902dd9666361348c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.450616 8544 raft_consensus.cc:359] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.450714 8544 raft_consensus.cc:385] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.450738 8544 raft_consensus.cc:740] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.450850 8544 consensus_queue.cc:260] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.451108 8544 ts_tablet_manager.cc:1434] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.451244 8544 tablet_bootstrap.cc:492] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.451979 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a28e8e2125f541a6944d94f73ee01cae" } new_table_name: "default.Table_e4f7aecd654a4a83b4435942625ca98a" modify_external_catalogs: true
I20251028 02:59:52.453538 8544 tablet_bootstrap.cc:654] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.453622 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_24016eb841cb46299d394b16f2d4f3b8" } modify_external_catalogs: true
I20251028 02:59:52.453713 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_24016eb841cb46299d394b16f2d4f3b8" } modify_external_catalogs: true
I20251028 02:59:52.454754 8552 tablet_replica.cc:333] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.454916 8552 raft_consensus.cc:2243] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.455056 8552 raft_consensus.cc:2272] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.456275 8552 ts_tablet_manager.cc:1916] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.460265 8552 ts_tablet_manager.cc:1929] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:52.460438 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 50a0a3d690154de7acd55f37ab736d13 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.460812 8552 log.cc:1199] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3e7dbafe267a4c62b0418d1b26585aad
I20251028 02:59:52.461331 8552 ts_tablet_manager.cc:1950] T 3e7dbafe267a4c62b0418d1b26585aad P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.462378 8552 tablet_replica.cc:333] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.462569 8552 raft_consensus.cc:2243] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:52.462792 8552 pending_rounds.cc:70] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:52.462888 8552 pending_rounds.cc:77] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597125836238848 op_type: NO_OP noop_request { }
I20251028 02:59:52.463008 8552 raft_consensus.cc:2889] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:52.463114 8552 raft_consensus.cc:2272] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.463806 8552 ts_tablet_manager.cc:1916] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.464807 8123 tablet_service.cc:1505] Processing CreateTablet for tablet a923402bbc734a0cb44aaf590d1e68ff (DEFAULT_TABLE table=default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.464870 8122 tablet_service.cc:1505] Processing CreateTablet for tablet aec9427ed4284d66b9bd46c0e1c8774c (DEFAULT_TABLE table=default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.464954 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 03959a61996b40d8b9ef8b4b9708f017 (DEFAULT_TABLE table=default.Table_b55d0b45f7e543e8a5927bde50d0502b [id=bc83d2963bb74c2d9b914dee53bcf6ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.465104 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a923402bbc734a0cb44aaf590d1e68ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.465108 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 68ca4ff164b24b7498f953ec11cf46be (DEFAULT_TABLE table=default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.465971 8398 tablet_service.cc:1505] Processing CreateTablet for tablet a923402bbc734a0cb44aaf590d1e68ff (DEFAULT_TABLE table=default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.466177 8263 tablet_service.cc:1505] Processing CreateTablet for tablet a923402bbc734a0cb44aaf590d1e68ff (DEFAULT_TABLE table=default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.466224 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a923402bbc734a0cb44aaf590d1e68ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.466255 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 02f6a32db68e4c609da39c3ee6c97348 (DEFAULT_TABLE table=default.Table_e69b7cea080549b1b8a67cb91703a472 [id=0c94092ef3844117af5e2aa5086dabb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.466398 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a923402bbc734a0cb44aaf590d1e68ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.466738 8401 tablet_service.cc:1505] Processing CreateTablet for tablet aec9427ed4284d66b9bd46c0e1c8774c (DEFAULT_TABLE table=default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.466972 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aec9427ed4284d66b9bd46c0e1c8774c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.467774 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3e7dbafe267a4c62b0418d1b26585aad (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:52.468567 8261 tablet_service.cc:1505] Processing CreateTablet for tablet aec9427ed4284d66b9bd46c0e1c8774c (DEFAULT_TABLE table=default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.468770 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aec9427ed4284d66b9bd46c0e1c8774c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.469321 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 02f6a32db68e4c609da39c3ee6c97348 (DEFAULT_TABLE table=default.Table_e69b7cea080549b1b8a67cb91703a472 [id=0c94092ef3844117af5e2aa5086dabb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.469547 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02f6a32db68e4c609da39c3ee6c97348. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.470822 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 68ca4ff164b24b7498f953ec11cf46be (DEFAULT_TABLE table=default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.470988 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68ca4ff164b24b7498f953ec11cf46be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.471661 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 02f6a32db68e4c609da39c3ee6c97348 (DEFAULT_TABLE table=default.Table_e69b7cea080549b1b8a67cb91703a472 [id=0c94092ef3844117af5e2aa5086dabb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.471899 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02f6a32db68e4c609da39c3ee6c97348. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.472967 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 03959a61996b40d8b9ef8b4b9708f017 (DEFAULT_TABLE table=default.Table_b55d0b45f7e543e8a5927bde50d0502b [id=bc83d2963bb74c2d9b914dee53bcf6ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.473192 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03959a61996b40d8b9ef8b4b9708f017. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.473977 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 68ca4ff164b24b7498f953ec11cf46be (DEFAULT_TABLE table=default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:52.474040 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df10bbbd757a4da8905718b61cbb10c8 consensus state has no leader
I20251028 02:59:52.474188 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68ca4ff164b24b7498f953ec11cf46be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.474232 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet aeddf70e5bd342f19f3f427d1df73814 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.475054 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 03959a61996b40d8b9ef8b4b9708f017 (DEFAULT_TABLE table=default.Table_b55d0b45f7e543e8a5927bde50d0502b [id=bc83d2963bb74c2d9b914dee53bcf6ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.475320 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03959a61996b40d8b9ef8b4b9708f017. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.478127 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03959a61996b40d8b9ef8b4b9708f017. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.478543 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68ca4ff164b24b7498f953ec11cf46be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.478847 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02f6a32db68e4c609da39c3ee6c97348. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.479220 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aec9427ed4284d66b9bd46c0e1c8774c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.484795 8546 tablet_bootstrap.cc:492] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.484903 8546 ts_tablet_manager.cc:1403] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20251028 02:59:52.485522 8546 raft_consensus.cc:359] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.485652 8546 raft_consensus.cc:385] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.485694 8546 raft_consensus.cc:740] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.485837 8546 consensus_queue.cc:260] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.486171 8546 ts_tablet_manager.cc:1434] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:52.486387 8546 tablet_bootstrap.cc:492] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.487468 8546 tablet_bootstrap.cc:654] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.490972 8545 tablet_bootstrap.cc:492] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.491098 8545 ts_tablet_manager.cc:1403] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.088s user 0.002s sys 0.000s
I20251028 02:59:52.491642 8545 raft_consensus.cc:359] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.491760 8545 raft_consensus.cc:385] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.491792 8545 raft_consensus.cc:740] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.491914 8545 consensus_queue.cc:260] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.492269 8545 ts_tablet_manager.cc:1434] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.492491 8545 tablet_bootstrap.cc:492] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.492501 8552 ts_tablet_manager.cc:1929] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.492575 8552 log.cc:1199] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c24384fecd7b4c0a9b2e3da1e0369866
I20251028 02:59:52.492864 8552 ts_tablet_manager.cc:1950] T c24384fecd7b4c0a9b2e3da1e0369866 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.495288 8545 tablet_bootstrap.cc:654] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.496863 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c24384fecd7b4c0a9b2e3da1e0369866 (table default.Table_dddd2e3cad6f4d96bd57539e6195677a [id=b82f549c8f03449eaa1762408bc67772]) successfully deleted
W20251028 02:59:52.497141 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 552c509b32374f5f9e3c232fbeddf9bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 552c509b32374f5f9e3c232fbeddf9bd already in progress: creating tablet
W20251028 02:59:52.497361 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 2d711941006d4417901135b7761a251f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d711941006d4417901135b7761a251f already in progress: creating tablet
I20251028 02:59:52.497427 8545 tablet_bootstrap.cc:492] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.497506 8545 ts_tablet_manager.cc:1403] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:52.498020 8545 raft_consensus.cc:359] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.498131 8545 raft_consensus.cc:385] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.498173 8545 raft_consensus.cc:740] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.498308 8545 consensus_queue.cc:260] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.498607 8545 ts_tablet_manager.cc:1434] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.498943 8545 tablet_bootstrap.cc:492] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.504045 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_309f970a07c6425bb6e764dfb593bc80" } modify_external_catalogs: true
I20251028 02:59:52.504200 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_309f970a07c6425bb6e764dfb593bc80" } modify_external_catalogs: true
I20251028 02:59:52.505198 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet 99968de685b44cbfa4c002a6c40e9cb3 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.506155 8938 raft_consensus.cc:493] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.506278 8938 raft_consensus.cc:515] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.506953 8938 leader_election.cc:290] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.506996 7968 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_310089f9c2b34a7e9e1c01c97fdb9b11 [id=c103fff3458d4d08affefb7c38d35e52] alter complete (version 1)
I20251028 02:59:52.507365 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_416a2058166f4d5a94419c7838d41b75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.507787 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_416a2058166f4d5a94419c7838d41b75 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.508553 7960 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_97b3a76d004f47c981178cf580a95985 [id=7d237f6c141e4d5e821305500a020dc0] alter complete (version 2)
I20251028 02:59:52.509549 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b88fda3f40540f3b42ee7f33dd33251" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.509897 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8b88fda3f40540f3b42ee7f33dd33251" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:52.510051 8356 leader_election.cc:343] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:52.510411 8354 leader_election.cc:343] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.510479 8354 leader_election.cc:304] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.510679 8938 raft_consensus.cc:2749] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:52.511282 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b88fda3f40540f3b42ee7f33dd33251 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.511504 8551 tablet_replica.cc:333] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.511634 8551 raft_consensus.cc:2243] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.511734 8551 raft_consensus.cc:2272] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.512254 8551 ts_tablet_manager.cc:1916] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.514657 8551 ts_tablet_manager.cc:1929] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.514957 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet a770dec97abb478b8c5504cc007b6092 with delete_type TABLET_DATA_DELETED (replaced by tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.515100 8551 log.cc:1199] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8b88fda3f40540f3b42ee7f33dd33251
I20251028 02:59:52.515165 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet bc61d4b98fa34f8c9aa0656aa43bc25c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.515198 8550 tablet_replica.cc:333] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.515323 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 6d94d536c1054613b2ae8ffcb08fbaab with delete_type TABLET_DATA_DELETED (replaced by tablet df10bbbd757a4da8905718b61cbb10c8) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.515367 8551 ts_tablet_manager.cc:1950] T 8b88fda3f40540f3b42ee7f33dd33251 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.515385 8550 raft_consensus.cc:2243] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.515501 8550 raft_consensus.cc:2272] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.515666 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 32ed23fd3cdc4be9ab0e65633c86e787 with delete_type TABLET_DATA_DELETED (replaced by tablet 2be7be7c732e4780859cbe6624edf54e) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.516093 8550 ts_tablet_manager.cc:1916] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.518129 8546 tablet_bootstrap.cc:492] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.518144 8550 ts_tablet_manager.cc:1929] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.518208 8550 log.cc:1199] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a770dec97abb478b8c5504cc007b6092
I20251028 02:59:52.518244 8546 ts_tablet_manager.cc:1403] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.032s user 0.000s sys 0.002s
I20251028 02:59:52.518550 8550 ts_tablet_manager.cc:1950] T a770dec97abb478b8c5504cc007b6092 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.518888 8546 raft_consensus.cc:359] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.519033 8546 raft_consensus.cc:385] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.519088 8546 raft_consensus.cc:740] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.519239 8546 consensus_queue.cc:260] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.519253 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a770dec97abb478b8c5504cc007b6092 (table default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]) successfully deleted
I20251028 02:59:52.519497 8550 tablet_replica.cc:333] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.519627 8550 raft_consensus.cc:2243] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.519731 8550 raft_consensus.cc:2272] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.520251 8550 ts_tablet_manager.cc:1916] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.521515 8546 ts_tablet_manager.cc:1434] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:52.522789 8550 ts_tablet_manager.cc:1929] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.522922 8550 log.cc:1199] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/bc61d4b98fa34f8c9aa0656aa43bc25c
I20251028 02:59:52.523218 8550 ts_tablet_manager.cc:1950] T bc61d4b98fa34f8c9aa0656aa43bc25c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.523973 8550 tablet_replica.cc:333] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.524161 8550 raft_consensus.cc:2243] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.524209 8546 tablet_bootstrap.cc:492] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.524354 8550 raft_consensus.cc:2272] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.524680 8545 tablet_bootstrap.cc:654] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.525015 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet bc61d4b98fa34f8c9aa0656aa43bc25c (table default.Table_4c1aacffea564931bb49e6bc29c6b0b7 [id=449d979448934b43a49f0b431b6dc45c]) successfully deleted
I20251028 02:59:52.525368 8546 tablet_bootstrap.cc:654] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.525866 8545 tablet_bootstrap.cc:492] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.525969 8545 ts_tablet_manager.cc:1403] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.002s
I20251028 02:59:52.526450 8545 raft_consensus.cc:359] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.526573 8545 raft_consensus.cc:385] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.526619 8545 raft_consensus.cc:740] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.526757 8545 consensus_queue.cc:260] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.527050 8545 ts_tablet_manager.cc:1434] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.527249 8545 tablet_bootstrap.cc:492] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.528105 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8b88fda3f40540f3b42ee7f33dd33251 (table default.Table_7fb744c8c91e4c9daadacdab1a408fa1 [id=98a07edd5f664674a2d33344038a8801]) successfully deleted
I20251028 02:59:52.528318 8545 tablet_bootstrap.cc:654] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.528712 8550 ts_tablet_manager.cc:1916] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.531057 8550 ts_tablet_manager.cc:1929] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.531177 8550 log.cc:1199] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/6d94d536c1054613b2ae8ffcb08fbaab
I20251028 02:59:52.531445 8550 ts_tablet_manager.cc:1950] T 6d94d536c1054613b2ae8ffcb08fbaab P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.531273 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet aeddf70e5bd342f19f3f427d1df73814
I20251028 02:59:52.531664 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 127e08303c564348b14257ec243418f4
I20251028 02:59:52.531847 7964 catalog_manager.cc:6719] ReplaceTablet: tablet aeddf70e5bd342f19f3f427d1df73814 deleted and replaced by tablet 5fd9cc1847ff4df7a09e2e698786d515
I20251028 02:59:52.532040 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 50a0a3d690154de7acd55f37ab736d13
I20251028 02:59:52.532159 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet aeddf70e5bd342f19f3f427d1df73814 with delete_type TABLET_DATA_DELETED (replaced by tablet 5fd9cc1847ff4df7a09e2e698786d515) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.532184 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 127e08303c564348b14257ec243418f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.532308 7966 catalog_manager.cc:6719] ReplaceTablet: tablet 50a0a3d690154de7acd55f37ab736d13 deleted and replaced by tablet 4ff8c974231f4b8aa18c18a05ad3e738
I20251028 02:59:52.532541 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 50a0a3d690154de7acd55f37ab736d13 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ff8c974231f4b8aa18c18a05ad3e738) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.532634 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet aeddf70e5bd342f19f3f427d1df73814 with delete_type TABLET_DATA_DELETED (replaced by tablet 5fd9cc1847ff4df7a09e2e698786d515) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.532855 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 127e08303c564348b14257ec243418f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.533219 8545 tablet_bootstrap.cc:492] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.533329 8545 ts_tablet_manager.cc:1403] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:52.533869 8545 raft_consensus.cc:359] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.533987 8545 raft_consensus.cc:385] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.534013 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 50a0a3d690154de7acd55f37ab736d13 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ff8c974231f4b8aa18c18a05ad3e738) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.534032 8545 raft_consensus.cc:740] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.534161 8545 consensus_queue.cc:260] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.534462 8545 ts_tablet_manager.cc:1434] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.534649 8545 tablet_bootstrap.cc:492] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.534683 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 50a0a3d690154de7acd55f37ab736d13 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ff8c974231f4b8aa18c18a05ad3e738) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:52.534827 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet aeddf70e5bd342f19f3f427d1df73814 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aeddf70e5bd342f19f3f427d1df73814 already in progress: creating tablet
I20251028 02:59:52.535000 8546 tablet_bootstrap.cc:492] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
W20251028 02:59:52.535046 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 127e08303c564348b14257ec243418f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 127e08303c564348b14257ec243418f4 already in progress: creating tablet
W20251028 02:59:52.535171 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 50a0a3d690154de7acd55f37ab736d13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50a0a3d690154de7acd55f37ab736d13 already in progress: creating tablet
I20251028 02:59:52.535383 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet aeddf70e5bd342f19f3f427d1df73814 with delete_type TABLET_DATA_DELETED (replaced by tablet 5fd9cc1847ff4df7a09e2e698786d515) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.535508 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 127e08303c564348b14257ec243418f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.535732 8550 tablet_replica.cc:333] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.535770 8546 ts_tablet_manager.cc:1403] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.001s
I20251028 02:59:52.535877 8550 raft_consensus.cc:2243] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.535984 8550 raft_consensus.cc:2272] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.536023 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 6d94d536c1054613b2ae8ffcb08fbaab (table default.Table_6b3cfd5f43ca4f3b82a37dab711251b0 [id=a6ae2be5375e4328848f28f43dfc49ab]) successfully deleted
I20251028 02:59:52.536309 8546 raft_consensus.cc:359] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.536417 8546 raft_consensus.cc:385] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.536449 8546 raft_consensus.cc:740] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.536531 8550 ts_tablet_manager.cc:1916] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.536561 8546 consensus_queue.cc:260] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.536859 8545 tablet_bootstrap.cc:654] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.539414 8550 ts_tablet_manager.cc:1929] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.539556 8550 log.cc:1199] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/32ed23fd3cdc4be9ab0e65633c86e787
I20251028 02:59:52.539853 8550 ts_tablet_manager.cc:1950] T 32ed23fd3cdc4be9ab0e65633c86e787 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.540323 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 32ed23fd3cdc4be9ab0e65633c86e787 (table default.Table_2a09149ed84d4343a81458e2fb78f262 [id=2bcc83cc472b4ec5a7ca79a8a7c5ff23]) successfully deleted
W20251028 02:59:52.541251 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet aeddf70e5bd342f19f3f427d1df73814 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aeddf70e5bd342f19f3f427d1df73814 already in progress: creating tablet
W20251028 02:59:52.541736 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 127e08303c564348b14257ec243418f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 127e08303c564348b14257ec243418f4 already in progress: creating tablet
I20251028 02:59:52.541859 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d4ddd31975814e4b9c829a132cdc6dad" } modify_external_catalogs: true
I20251028 02:59:52.541977 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d4ddd31975814e4b9c829a132cdc6dad" } modify_external_catalogs: true
W20251028 02:59:52.542080 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 50a0a3d690154de7acd55f37ab736d13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50a0a3d690154de7acd55f37ab736d13 already in progress: creating tablet
W20251028 02:59:52.542168 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 50a0a3d690154de7acd55f37ab736d13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50a0a3d690154de7acd55f37ab736d13 already in progress: creating tablet
W20251028 02:59:52.542202 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet aeddf70e5bd342f19f3f427d1df73814 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aeddf70e5bd342f19f3f427d1df73814 already in progress: creating tablet
W20251028 02:59:52.542311 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 127e08303c564348b14257ec243418f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 127e08303c564348b14257ec243418f4 already in progress: creating tablet
I20251028 02:59:52.542399 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_12db05bb2e3a40d594ac678a0409bf5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.542797 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12db05bb2e3a40d594ac678a0409bf5d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.542881 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0ac5b50f10cd4c52b002d3906b21b3a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.543213 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ac5b50f10cd4c52b002d3906b21b3a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.543432 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c2f3577aa7544c82997c62b45e5bb823" } new_table_name: "default.Table_a1cf1772f69f4e4284a15ea70a3407d6" modify_external_catalogs: true
I20251028 02:59:52.545475 8546 ts_tablet_manager.cc:1434] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20251028 02:59:52.546700 8544 tablet_bootstrap.cc:492] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.546804 8544 ts_tablet_manager.cc:1403] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.096s user 0.002s sys 0.000s
I20251028 02:59:52.547289 8544 raft_consensus.cc:359] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.548082 8544 raft_consensus.cc:385] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.548120 8544 raft_consensus.cc:740] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.548249 8544 consensus_queue.cc:260] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.548533 8544 ts_tablet_manager.cc:1434] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:52.548758 8544 tablet_bootstrap.cc:492] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.549638 8544 tablet_bootstrap.cc:654] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.550060 8546 tablet_bootstrap.cc:492] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.551065 8544 tablet_bootstrap.cc:492] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.551151 8544 ts_tablet_manager.cc:1403] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:52.551589 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 83495009a1dc4f4388d2ae65f2dda548
I20251028 02:59:52.551657 8544 raft_consensus.cc:359] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.551753 8544 raft_consensus.cc:385] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.551785 8544 raft_consensus.cc:740] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.552495 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 83495009a1dc4f4388d2ae65f2dda548 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.552837 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 83495009a1dc4f4388d2ae65f2dda548 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.552914 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_013e772cd60943509e2ea1c830dfe151" } new_table_name: "default.Table_726e0c2d9d4044d8b79bb266e21a8d9c" modify_external_catalogs: true
W20251028 02:59:52.552983 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 83495009a1dc4f4388d2ae65f2dda548 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83495009a1dc4f4388d2ae65f2dda548 already in progress: creating tablet
I20251028 02:59:52.553016 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet db6eadc5927d4876af9a50b44be97a63 from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:52.553679 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 83495009a1dc4f4388d2ae65f2dda548 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83495009a1dc4f4388d2ae65f2dda548 already in progress: creating tablet
I20251028 02:59:52.553707 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 99968de685b44cbfa4c002a6c40e9cb3
I20251028 02:59:52.553699 8544 consensus_queue.cc:260] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.553938 7963 catalog_manager.cc:6719] ReplaceTablet: tablet 99968de685b44cbfa4c002a6c40e9cb3 deleted and replaced by tablet 1e9234b72f4448089fe7ff6ba49483b7
I20251028 02:59:52.554028 8544 ts_tablet_manager.cc:1434] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:52.554224 8544 tablet_bootstrap.cc:492] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.554319 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 99968de685b44cbfa4c002a6c40e9cb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e9234b72f4448089fe7ff6ba49483b7) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.554497 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 99968de685b44cbfa4c002a6c40e9cb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e9234b72f4448089fe7ff6ba49483b7) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:52.554916 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 99968de685b44cbfa4c002a6c40e9cb3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99968de685b44cbfa4c002a6c40e9cb3 already in progress: creating tablet
W20251028 02:59:52.555060 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 99968de685b44cbfa4c002a6c40e9cb3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99968de685b44cbfa4c002a6c40e9cb3 already in progress: creating tablet
I20251028 02:59:52.556442 8544 tablet_bootstrap.cc:654] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.556718 8556 raft_consensus.cc:493] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.556829 8556 raft_consensus.cc:515] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.557210 8556 leader_election.cc:290] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.557561 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3410b1b517b146d4b38cb47862c85e68" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.557602 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3410b1b517b146d4b38cb47862c85e68" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.557721 8420 raft_consensus.cc:2468] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
W20251028 02:59:52.557948 8218 leader_election.cc:343] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.558043 8220 leader_election.cc:304] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.558801 8546 tablet_bootstrap.cc:654] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.558811 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 83495009a1dc4f4388d2ae65f2dda548 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.558966 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 909b08e0860b4e4f9bb1822acfd1dc02 with delete_type TABLET_DATA_DELETED (replaced by tablet 8b88fda3f40540f3b42ee7f33dd33251) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.559067 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 99968de685b44cbfa4c002a6c40e9cb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e9234b72f4448089fe7ff6ba49483b7) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.560611 8283 raft_consensus.cc:1275] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:52.561349 8545 tablet_bootstrap.cc:492] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.561437 8545 ts_tablet_manager.cc:1403] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20251028 02:59:52.561924 8545 raft_consensus.cc:359] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.561982 8549 consensus_queue.cc:1048] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.562775 8556 raft_consensus.cc:2804] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.562834 8556 raft_consensus.cc:493] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.562870 8556 raft_consensus.cc:3060] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.563268 8546 tablet_bootstrap.cc:492] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.563375 8546 ts_tablet_manager.cc:1403] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.001s
I20251028 02:59:52.563822 8546 raft_consensus.cc:359] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.563917 8546 raft_consensus.cc:385] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.563944 8556 raft_consensus.cc:515] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.564297 8556 leader_election.cc:290] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.564759 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3410b1b517b146d4b38cb47862c85e68" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.564894 8420 raft_consensus.cc:3060] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.565440 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet d928418826a84f63803ae474dc05de7e with delete_type TABLET_DATA_DELETED (replaced by tablet da1563d65b4d4ab48325a4f08ee89ed5) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.565716 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3410b1b517b146d4b38cb47862c85e68" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.565822 8420 raft_consensus.cc:2468] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:52.566156 8220 leader_election.cc:304] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.562033 8545 raft_consensus.cc:385] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.566651 8545 raft_consensus.cc:740] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.566807 8545 consensus_queue.cc:260] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:52.566953 8218 leader_election.cc:343] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.567088 8545 ts_tablet_manager.cc:1434] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251028 02:59:52.567132 8148 raft_consensus.cc:1275] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:52.567265 8545 tablet_bootstrap.cc:492] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.567447 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d9729143013a4ead9c3144747ecbe58c
I20251028 02:59:52.568141 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet d9729143013a4ead9c3144747ecbe58c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.568190 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet d9729143013a4ead9c3144747ecbe58c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.568248 8938 consensus_queue.cc:1048] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.568383 8545 tablet_bootstrap.cc:654] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.568485 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet d9729143013a4ead9c3144747ecbe58c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.568647 8552 tablet_replica.cc:333] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.568787 8552 raft_consensus.cc:2243] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.568898 8552 raft_consensus.cc:2272] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.569478 8552 ts_tablet_manager.cc:1916] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.570011 8553 raft_consensus.cc:2804] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.570123 8553 raft_consensus.cc:697] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:52.570246 8553 consensus_queue.cc:237] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.571652 8551 tablet_replica.cc:333] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.571789 8551 raft_consensus.cc:2243] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.571930 8551 raft_consensus.cc:2272] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.573012 8753 tablet.cc:1722] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:52.573163 8550 tablet_replica.cc:333] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.573280 8550 raft_consensus.cc:2243] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.573382 8550 raft_consensus.cc:2272] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.573765 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 63ee5aa5a6e841fdb9244563d75394ac (table default.Table_a1cf1772f69f4e4284a15ea70a3407d6 [id=79b638b293df431cbd580e62d24e8238]). Expected version 1 got 0
W20251028 02:59:52.574954 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 83495009a1dc4f4388d2ae65f2dda548 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83495009a1dc4f4388d2ae65f2dda548 already in progress: creating tablet
I20251028 02:59:52.575711 8550 ts_tablet_manager.cc:1916] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.576153 8551 ts_tablet_manager.cc:1916] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.576444 8884 tablet.cc:1722] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:52.577704 8965 raft_consensus.cc:493] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.577806 8965 raft_consensus.cc:515] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.578233 8965 leader_election.cc:290] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.578258 8551 ts_tablet_manager.cc:1929] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.578326 8551 log.cc:1199] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/d9729143013a4ead9c3144747ecbe58c
I20251028 02:59:52.578579 8551 ts_tablet_manager.cc:1950] T d9729143013a4ead9c3144747ecbe58c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.578639 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "909b08e0860b4e4f9bb1822acfd1dc02" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.578925 8552 ts_tablet_manager.cc:1929] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20251028 02:59:52.579102 8218 leader_election.cc:343] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: Tablet not RUNNING: STOPPED
I20251028 02:59:52.580730 8550 ts_tablet_manager.cc:1929] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.581243 8550 log.cc:1199] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/909b08e0860b4e4f9bb1822acfd1dc02
I20251028 02:59:52.581449 8550 ts_tablet_manager.cc:1950] T 909b08e0860b4e4f9bb1822acfd1dc02 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.581589 8552 log.cc:1199] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/d9729143013a4ead9c3144747ecbe58c
I20251028 02:59:52.582141 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 909b08e0860b4e4f9bb1822acfd1dc02 (table default.Table_7fb744c8c91e4c9daadacdab1a408fa1 [id=98a07edd5f664674a2d33344038a8801]) successfully deleted
I20251028 02:59:52.582616 8552 ts_tablet_manager.cc:1950] T d9729143013a4ead9c3144747ecbe58c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.583379 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 63ee5aa5a6e841fdb9244563d75394ac from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.563951 8546 raft_consensus.cc:740] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.583781 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet d9729143013a4ead9c3144747ecbe58c (table default.Table_d4ddd31975814e4b9c829a132cdc6dad [id=f714f168dbd94c4aaf69df7cb6e4bb3a]) successfully deleted
I20251028 02:59:52.583779 8546 consensus_queue.cc:260] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:52.584400 8083 consensus_peers.cc:597] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d9729143013a4ead9c3144747ecbe58c. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:52.584510 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet d9729143013a4ead9c3144747ecbe58c (table default.Table_d4ddd31975814e4b9c829a132cdc6dad [id=f714f168dbd94c4aaf69df7cb6e4bb3a]) successfully deleted
I20251028 02:59:52.584936 8546 ts_tablet_manager.cc:1434] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.022s user 0.001s sys 0.000s
I20251028 02:59:52.585103 8546 tablet_bootstrap.cc:492] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:52.585944 8220 leader_election.cc:343] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 909b08e0860b4e4f9bb1822acfd1dc02
I20251028 02:59:52.586038 8220 leader_election.cc:304] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.586192 8550 tablet_replica.cc:333] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.586319 8550 raft_consensus.cc:2243] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.586349 8965 raft_consensus.cc:2749] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:52.586481 8550 raft_consensus.cc:2272] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.586414 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_23ea93791ab84db8acf0b4e6c21141d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.586830 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_23ea93791ab84db8acf0b4e6c21141d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.587266 8546 tablet_bootstrap.cc:654] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:52.587801 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 99968de685b44cbfa4c002a6c40e9cb3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99968de685b44cbfa4c002a6c40e9cb3 already in progress: creating tablet
I20251028 02:59:52.588318 8550 ts_tablet_manager.cc:1916] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:52.589825 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e359f0121c334cc1a8b3eedbbe64bda2 consensus state has no leader
I20251028 02:59:52.590126 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet db6eadc5927d4876af9a50b44be97a63
I20251028 02:59:52.590394 7968 catalog_manager.cc:6719] ReplaceTablet: tablet db6eadc5927d4876af9a50b44be97a63 deleted and replaced by tablet 67c6ceefa3294af8b68150609f46c117
I20251028 02:59:52.590759 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet db6eadc5927d4876af9a50b44be97a63 with delete_type TABLET_DATA_DELETED (replaced by tablet 67c6ceefa3294af8b68150609f46c117) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.590862 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet db6eadc5927d4876af9a50b44be97a63 with delete_type TABLET_DATA_DELETED (replaced by tablet 67c6ceefa3294af8b68150609f46c117) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:52.591208 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet db6eadc5927d4876af9a50b44be97a63 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet db6eadc5927d4876af9a50b44be97a63 already in progress: creating tablet
I20251028 02:59:52.591396 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet db6eadc5927d4876af9a50b44be97a63 with delete_type TABLET_DATA_DELETED (replaced by tablet 67c6ceefa3294af8b68150609f46c117) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.592139 8545 tablet_bootstrap.cc:492] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.592231 8545 ts_tablet_manager.cc:1403] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20251028 02:59:52.592795 8545 raft_consensus.cc:359] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:52.592881 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet db6eadc5927d4876af9a50b44be97a63 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet db6eadc5927d4876af9a50b44be97a63 already in progress: creating tablet
I20251028 02:59:52.592916 8545 raft_consensus.cc:385] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.592949 8545 raft_consensus.cc:740] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.593091 8545 consensus_queue.cc:260] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.593746 8550 ts_tablet_manager.cc:1929] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.594261 8550 log.cc:1199] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/d928418826a84f63803ae474dc05de7e
I20251028 02:59:52.594321 8965 consensus_queue.cc:1048] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.594547 8550 ts_tablet_manager.cc:1950] T d928418826a84f63803ae474dc05de7e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.594820 8545 ts_tablet_manager.cc:1434] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:52.595009 8545 tablet_bootstrap.cc:492] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.595242 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet d928418826a84f63803ae474dc05de7e (table default.Table_27a9af4382b34a3eab870c4d40a6c5bf [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
I20251028 02:59:52.595496 8550 tablet_replica.cc:333] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.595672 8550 raft_consensus.cc:2243] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:52.595907 8550 raft_consensus.cc:2272] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.596499 8550 ts_tablet_manager.cc:1916] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.596824 8545 tablet_bootstrap.cc:654] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.599171 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 63ee5aa5a6e841fdb9244563d75394ac
I20251028 02:59:52.599509 7961 catalog_manager.cc:6719] ReplaceTablet: tablet 63ee5aa5a6e841fdb9244563d75394ac deleted and replaced by tablet a8331b21af69463cb7589619e784051d
I20251028 02:59:52.599773 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 63ee5aa5a6e841fdb9244563d75394ac with delete_type TABLET_DATA_DELETED (replaced by tablet a8331b21af69463cb7589619e784051d) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.599916 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 63ee5aa5a6e841fdb9244563d75394ac with delete_type TABLET_DATA_DELETED (replaced by tablet a8331b21af69463cb7589619e784051d) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.600090 8552 tablet_replica.cc:333] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.600224 8552 raft_consensus.cc:2243] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.600342 8552 raft_consensus.cc:2272] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.600975 8551 tablet_replica.cc:333] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.601114 8551 raft_consensus.cc:2243] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:52.601270 8551 raft_consensus.cc:2272] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.601938 8551 ts_tablet_manager.cc:1916] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.604285 8551 ts_tablet_manager.cc:1929] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:52.604353 8551 log.cc:1199] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/63ee5aa5a6e841fdb9244563d75394ac
I20251028 02:59:52.604593 8551 ts_tablet_manager.cc:1950] T 63ee5aa5a6e841fdb9244563d75394ac P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.606114 8648 raft_consensus.cc:493] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.606213 8648 raft_consensus.cc:515] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.606599 8648 leader_election.cc:290] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.607152 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d248313db35e40e38e2c5e07e68a69af" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.607285 8283 raft_consensus.cc:2468] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:52.607623 8356 leader_election.cc:304] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.607905 8648 raft_consensus.cc:2804] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.607962 8648 raft_consensus.cc:493] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.608008 8648 raft_consensus.cc:3060] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.608085 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 63ee5aa5a6e841fdb9244563d75394ac with delete_type TABLET_DATA_DELETED (replaced by tablet a8331b21af69463cb7589619e784051d) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.609057 8648 raft_consensus.cc:515] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.609386 8648 leader_election.cc:290] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.609787 8550 ts_tablet_manager.cc:1929] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.609825 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 63ee5aa5a6e841fdb9244563d75394ac (table default.Table_a1cf1772f69f4e4284a15ea70a3407d6 [id=79b638b293df431cbd580e62d24e8238]) successfully deleted
I20251028 02:59:52.609925 8550 log.cc:1199] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/d9729143013a4ead9c3144747ecbe58c
I20251028 02:59:52.610190 8550 ts_tablet_manager.cc:1950] T d9729143013a4ead9c3144747ecbe58c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.610483 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 002de8aede994720b8163f41938af6c4 (DEFAULT_TABLE table=default.Table_c3d6504130c54bdeade5bb23b6cd0679 [id=371c4929d59942f79f41ad071b61e017]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.610777 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 002de8aede994720b8163f41938af6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.611078 8550 tablet_replica.cc:333] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.611207 8550 raft_consensus.cc:2243] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.611286 8550 pending_rounds.cc:70] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 2 pending ops.
I20251028 02:59:52.611338 8550 pending_rounds.cc:77] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7215597127922200576 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "63ee5aa5a6e841fdb9244563d75394ac" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_a1cf1772f69f4e4284a15ea70a3407d6" dest_uuid: "d7e4cf694f71438298dd32a346abcf75" new_extra_config { } }
I20251028 02:59:52.611548 8550 pending_rounds.cc:77] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597126001627136 op_type: NO_OP noop_request { }
I20251028 02:59:52.611591 8550 raft_consensus.cc:2889] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:52.611640 8550 raft_consensus.cc:2272] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.611819 7967 catalog_manager.cc:5649] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:52.612035 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 (table default.Table_7778bd93ae56485f921a20d0df78a3ec [id=7a2a6ab3c9744942a9413c6cb057f527]). Expected version 4 got 0
I20251028 02:59:52.612151 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 71feaa313a8f45798561741f7b5d9b20 (table default.Table_d5e26eda7deb4e4f9d3a93ac052b01c9 [id=5581f43ea3854dbeba821353ffce6ec7]). Expected version 2 got 0
I20251028 02:59:52.612304 8546 tablet_bootstrap.cc:492] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.612382 8546 ts_tablet_manager.cc:1403] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.001s
I20251028 02:59:52.612826 8546 raft_consensus.cc:359] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.612923 8546 raft_consensus.cc:385] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.612953 8546 raft_consensus.cc:740] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.613063 8546 consensus_queue.cc:260] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.613338 8546 ts_tablet_manager.cc:1434] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:52.613520 8546 tablet_bootstrap.cc:492] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.613687 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet d9729143013a4ead9c3144747ecbe58c (table default.Table_d4ddd31975814e4b9c829a132cdc6dad [id=f714f168dbd94c4aaf69df7cb6e4bb3a]) successfully deleted
W20251028 02:59:52.613869 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet db6eadc5927d4876af9a50b44be97a63 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet db6eadc5927d4876af9a50b44be97a63 already in progress: creating tablet
I20251028 02:59:52.614163 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 6c87d03fad194ca89f1808da6dbdf0a4 (DEFAULT_TABLE table=default.Table_bcdd1e4fff1a45e39b03d0c7f2268672 [id=a6d84958fd654087940c9806e2e3fbc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.614197 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 7f804f81d70749b5bdefeca4e0bad6fe (DEFAULT_TABLE table=default.Table_587714a762d0451d98e0b71b42c3834c [id=eb0e103661fc4bc7b27bb78fc333fd56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.614346 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d248313db35e40e38e2c5e07e68a69af" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:52.614406 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c87d03fad194ca89f1808da6dbdf0a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.614459 8283 raft_consensus.cc:3060] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.614466 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f804f81d70749b5bdefeca4e0bad6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.615008 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 6c87d03fad194ca89f1808da6dbdf0a4 (DEFAULT_TABLE table=default.Table_bcdd1e4fff1a45e39b03d0c7f2268672 [id=a6d84958fd654087940c9806e2e3fbc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.615279 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c87d03fad194ca89f1808da6dbdf0a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.615272 8546 tablet_bootstrap.cc:654] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.615962 8283 raft_consensus.cc:2468] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:52.616456 8546 tablet_bootstrap.cc:492] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.616482 8356 leader_election.cc:304] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.616526 8546 ts_tablet_manager.cc:1403] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:52.616814 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 002de8aede994720b8163f41938af6c4 (DEFAULT_TABLE table=default.Table_c3d6504130c54bdeade5bb23b6cd0679 [id=371c4929d59942f79f41ad071b61e017]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.617064 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 002de8aede994720b8163f41938af6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.617026 8546 raft_consensus.cc:359] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.616995 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b883c2cd0a0f49fca261f85e62e9bc18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:52.617121 8546 raft_consensus.cc:385] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.617156 8546 raft_consensus.cc:740] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.617275 8546 consensus_queue.cc:260] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:52.617421 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b883c2cd0a0f49fca261f85e62e9bc18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.617559 8546 ts_tablet_manager.cc:1434] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.617722 8546 tablet_bootstrap.cc:492] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.617794 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 6c87d03fad194ca89f1808da6dbdf0a4 (DEFAULT_TABLE table=default.Table_bcdd1e4fff1a45e39b03d0c7f2268672 [id=a6d84958fd654087940c9806e2e3fbc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.618074 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c87d03fad194ca89f1808da6dbdf0a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.618533 8616 raft_consensus.cc:2804] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.618714 8938 raft_consensus.cc:493] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.618753 8546 tablet_bootstrap.cc:654] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.618773 8938 raft_consensus.cc:515] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.618974 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 002de8aede994720b8163f41938af6c4 (DEFAULT_TABLE table=default.Table_c3d6504130c54bdeade5bb23b6cd0679 [id=371c4929d59942f79f41ad071b61e017]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.619184 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 002de8aede994720b8163f41938af6c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.619182 8938 leader_election.cc:290] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.618654 8616 raft_consensus.cc:697] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:52.619658 8616 consensus_queue.cc:237] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.619786 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 7f804f81d70749b5bdefeca4e0bad6fe (DEFAULT_TABLE table=default.Table_587714a762d0451d98e0b71b42c3834c [id=eb0e103661fc4bc7b27bb78fc333fd56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.620033 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f804f81d70749b5bdefeca4e0bad6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.621316 8546 tablet_bootstrap.cc:492] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.621392 8546 ts_tablet_manager.cc:1403] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20251028 02:59:52.621951 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d248313db35e40e38e2c5e07e68a69af" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.622066 8546 raft_consensus.cc:359] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.622179 8546 raft_consensus.cc:385] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.622207 8546 raft_consensus.cc:740] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.622218 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d248313db35e40e38e2c5e07e68a69af" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.622330 8546 consensus_queue.cc:260] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.622715 8556 raft_consensus.cc:493] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.622812 8556 raft_consensus.cc:515] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.623148 8556 leader_election.cc:290] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.623924 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_38f86582ea6e4e5fb73330ab8f1d4c25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.624418 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38f86582ea6e4e5fb73330ab8f1d4c25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.625002 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_cbf114f7a2674dfa956f0f5d163c7622" } modify_external_catalogs: true
I20251028 02:59:52.625116 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_cbf114f7a2674dfa956f0f5d163c7622" } modify_external_catalogs: true
I20251028 02:59:52.625121 8546 ts_tablet_manager.cc:1434] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:52.625312 8546 tablet_bootstrap.cc:492] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.625352 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8cb97d5192684d748b8b485f891b0dc0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.625695 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_83adc71d84344ff686fc777f2f72d54b" } modify_external_catalogs: true
I20251028 02:59:52.625793 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_83adc71d84344ff686fc777f2f72d54b" } modify_external_catalogs: true
I20251028 02:59:52.626304 8546 tablet_bootstrap.cc:654] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:52.627090 8218 leader_election.cc:343] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.627513 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8cb97d5192684d748b8b485f891b0dc0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.627633 8421 raft_consensus.cc:2468] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
W20251028 02:59:52.627676 8354 leader_election.cc:343] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:52.627863 8354 leader_election.cc:343] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.628105 8220 leader_election.cc:304] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.628185 8616 raft_consensus.cc:493] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.628264 8616 raft_consensus.cc:515] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.628377 8556 raft_consensus.cc:2804] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.628423 8556 raft_consensus.cc:493] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.628458 8556 raft_consensus.cc:3060] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.628610 8616 leader_election.cc:290] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.629015 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71feaa313a8f45798561741f7b5d9b20" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.629421 8556 raft_consensus.cc:515] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.629572 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488bc52f1219478ab8843c9b1d79de2b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.629786 8556 leader_election.cc:290] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:52.630141 8356 leader_election.cc:343] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.630364 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71feaa313a8f45798561741f7b5d9b20" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.630493 8283 raft_consensus.cc:2468] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:52.630740 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8cb97d5192684d748b8b485f891b0dc0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.630851 8421 raft_consensus.cc:3060] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.631009 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8cb97d5192684d748b8b485f891b0dc0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.631415 8356 leader_election.cc:304] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.631580 8616 raft_consensus.cc:2804] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.631628 8616 raft_consensus.cc:493] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.631665 8616 raft_consensus.cc:3060] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.631798 8421 raft_consensus.cc:2468] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
W20251028 02:59:52.632371 8218 leader_election.cc:343] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.632339 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2ad18e95c3a14e039a63bebaa46268fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:52.632586 8220 leader_election.cc:304] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.632639 8616 raft_consensus.cc:515] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:52.632794 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ad18e95c3a14e039a63bebaa46268fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.632797 8556 raft_consensus.cc:2804] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.632882 8556 raft_consensus.cc:697] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:52.632975 8616 leader_election.cc:290] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.633002 8556 consensus_queue.cc:237] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:52.633636 8354 leader_election.cc:343] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.634109 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71feaa313a8f45798561741f7b5d9b20" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.634307 8546 tablet_bootstrap.cc:492] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.634393 8546 ts_tablet_manager.cc:1403] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20251028 02:59:52.634406 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488bc52f1219478ab8843c9b1d79de2b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.634840 8546 raft_consensus.cc:359] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.634932 8546 raft_consensus.cc:385] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.634958 8546 raft_consensus.cc:740] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.634946 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7f5ff2cc8eef4fa28d84a14d3851d912" } new_table_name: "default.Table_8fdeffc6f6234305a829715139f150db" modify_external_catalogs: true
I20251028 02:59:52.635084 8546 consensus_queue.cc:260] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.635353 8546 ts_tablet_manager.cc:1434] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.636124 8546 tablet_bootstrap.cc:492] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.636267 8146 raft_consensus.cc:3060] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.637113 8546 tablet_bootstrap.cc:654] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.637635 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "71feaa313a8f45798561741f7b5d9b20" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:52.637749 8283 raft_consensus.cc:3060] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.639118 8283 raft_consensus.cc:2468] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:52.639787 8356 leader_election.cc:304] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.640000 8648 raft_consensus.cc:2804] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.640094 8648 raft_consensus.cc:697] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:52.640247 8648 consensus_queue.cc:237] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:52.641319 8354 leader_election.cc:343] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:52.641525 8354 leader_election.cc:343] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.641584 8354 leader_election.cc:304] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.641597 8556 consensus_queue.cc:1048] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.642354 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 7f804f81d70749b5bdefeca4e0bad6fe (DEFAULT_TABLE table=default.Table_587714a762d0451d98e0b71b42c3834c [id=eb0e103661fc4bc7b27bb78fc333fd56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.642628 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f804f81d70749b5bdefeca4e0bad6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.644235 8546 tablet_bootstrap.cc:492] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.644376 8546 ts_tablet_manager.cc:1403] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:52.644963 8546 raft_consensus.cc:359] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.645133 8546 raft_consensus.cc:385] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.645200 8546 raft_consensus.cc:740] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.645376 8546 consensus_queue.cc:260] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.646644 8546 ts_tablet_manager.cc:1434] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:52.646878 8546 tablet_bootstrap.cc:492] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.648718 7964 catalog_manager.cc:5931] Not sending DeleteTablet requests; no consensus state for tablet 87e33da4c79f498d9ed02400a84f8434
I20251028 02:59:52.648955 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 0141ebd1ddad421bbb8eca7615e78351
I20251028 02:59:52.649015 8546 tablet_bootstrap.cc:654] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.649943 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet a770dec97abb478b8c5504cc007b6092 with delete_type TABLET_DATA_DELETED (replaced by tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.649963 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 909b08e0860b4e4f9bb1822acfd1dc02 with delete_type TABLET_DATA_DELETED (replaced by tablet 8b88fda3f40540f3b42ee7f33dd33251) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.650295 8648 raft_consensus.cc:2749] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:52.650301 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 0141ebd1ddad421bbb8eca7615e78351 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.650491 8551 tablet_replica.cc:333] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.650529 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 0141ebd1ddad421bbb8eca7615e78351 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.650611 8551 raft_consensus.cc:2243] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.650710 8551 raft_consensus.cc:2272] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.651145 8551 ts_tablet_manager.cc:1916] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.651218 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 0141ebd1ddad421bbb8eca7615e78351 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.653447 8551 ts_tablet_manager.cc:1929] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.653522 8551 log.cc:1199] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/0141ebd1ddad421bbb8eca7615e78351
I20251028 02:59:52.653713 8551 ts_tablet_manager.cc:1950] T 0141ebd1ddad421bbb8eca7615e78351 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.653769 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet cfd68d91fd9b4c69871383e90bd2ff85 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.654245 8551 tablet_replica.cc:333] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.654363 8551 raft_consensus.cc:2243] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.654500 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 0141ebd1ddad421bbb8eca7615e78351 (table default.Table_83adc71d84344ff686fc777f2f72d54b [id=0b22f43d0bda44b98e6f4b2f26153c5c]) successfully deleted
I20251028 02:59:52.656473 8551 raft_consensus.cc:2272] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.656991 8551 ts_tablet_manager.cc:1916] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.659220 8551 ts_tablet_manager.cc:1929] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.659384 8551 log.cc:1199] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/cfd68d91fd9b4c69871383e90bd2ff85
I20251028 02:59:52.659652 8551 ts_tablet_manager.cc:1950] T cfd68d91fd9b4c69871383e90bd2ff85 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.659821 8753 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.663542 8726 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.668442 8884 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.662829 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet cfd68d91fd9b4c69871383e90bd2ff85 (table default.Table_620185a247324af6b2054263457ec7da [id=c33a0072d1384953b73c2e3d90176cc4]) successfully deleted
I20251028 02:59:52.677233 8552 ts_tablet_manager.cc:1916] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:52.677965 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0703e18d7dba4d628e3cc310be55fcf5 consensus state has no leader
I20251028 02:59:52.678056 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e95e72f2a7164cdd902dd9666361348c" } new_table_name: "default.Table_640f9dd799c74ce4bbbc0839fd52b04f" modify_external_catalogs: true
W20251028 02:59:52.678540 8990 consensus_queue.cc:1175] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20251028 02:59:52.679677 8550 ts_tablet_manager.cc:1916] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.682092 8550 ts_tablet_manager.cc:1929] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:52.682171 8550 log.cc:1199] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/63ee5aa5a6e841fdb9244563d75394ac
I20251028 02:59:52.682437 8550 ts_tablet_manager.cc:1950] T 63ee5aa5a6e841fdb9244563d75394ac P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.683231 8545 tablet_bootstrap.cc:492] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.683353 8545 ts_tablet_manager.cc:1403] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.088s user 0.002s sys 0.000s
I20251028 02:59:52.683624 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 63ee5aa5a6e841fdb9244563d75394ac (table default.Table_a1cf1772f69f4e4284a15ea70a3407d6 [id=79b638b293df431cbd580e62d24e8238]) successfully deleted
W20251028 02:59:52.683813 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 0141ebd1ddad421bbb8eca7615e78351 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0141ebd1ddad421bbb8eca7615e78351 already in progress: creating tablet
I20251028 02:59:52.683876 8545 raft_consensus.cc:359] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.683996 8545 raft_consensus.cc:385] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.684033 8545 raft_consensus.cc:740] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.684166 8545 consensus_queue.cc:260] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.684429 8545 ts_tablet_manager.cc:1434] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.684634 8545 tablet_bootstrap.cc:492] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.684780 8552 ts_tablet_manager.cc:1929] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:52.684857 8552 log.cc:1199] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/63ee5aa5a6e841fdb9244563d75394ac
I20251028 02:59:52.685113 8552 ts_tablet_manager.cc:1950] T 63ee5aa5a6e841fdb9244563d75394ac P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.685731 8552 tablet_replica.cc:333] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.685738 8545 tablet_bootstrap.cc:654] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.685873 8552 raft_consensus.cc:2243] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.685992 8552 raft_consensus.cc:2272] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.687129 8544 tablet_bootstrap.cc:492] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.687234 8544 ts_tablet_manager.cc:1403] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.133s user 0.002s sys 0.000s
I20251028 02:59:52.687785 8544 raft_consensus.cc:359] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.687886 8544 raft_consensus.cc:385] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.687917 8544 raft_consensus.cc:740] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.688038 8544 consensus_queue.cc:260] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.688319 8556 consensus_queue.cc:1048] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.688388 8544 ts_tablet_manager.cc:1434] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.688585 8544 tablet_bootstrap.cc:492] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.688916 8545 tablet_bootstrap.cc:492] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.689000 8545 ts_tablet_manager.cc:1403] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:52.689536 8545 raft_consensus.cc:359] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.689657 8545 raft_consensus.cc:385] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.689689 8545 raft_consensus.cc:740] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.689693 8544 tablet_bootstrap.cc:654] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.689847 8545 consensus_queue.cc:260] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.690116 8146 raft_consensus.cc:3060] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.690616 8545 ts_tablet_manager.cc:1434] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:52.690820 8545 tablet_bootstrap.cc:492] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.691141 8552 ts_tablet_manager.cc:1916] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.691594 8556 consensus_queue.cc:1048] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.691712 7969 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_e4f7aecd654a4a83b4435942625ca98a [id=4eb2743632814baab2e39bbdce700d6b] alter complete (version 2)
I20251028 02:59:52.691897 8545 tablet_bootstrap.cc:654] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:52.692157 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 50e6d41d1cf746aab2802b5c6b2d58b3 has no consensus state
I20251028 02:59:52.692273 8546 tablet_bootstrap.cc:492] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.692353 8546 ts_tablet_manager.cc:1403] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.046s user 0.000s sys 0.002s
I20251028 02:59:52.693325 7961 catalog_manager.cc:5649] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:52.693462 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 63ee5aa5a6e841fdb9244563d75394ac (table default.Table_a1cf1772f69f4e4284a15ea70a3407d6 [id=79b638b293df431cbd580e62d24e8238]) successfully deleted
I20251028 02:59:52.693652 7961 catalog_manager.cc:5649] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:52.693864 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 8cb97d5192684d748b8b485f891b0dc0 (table default.Table_8bd9c3af53644cbcb68461ae9433a9f1 [id=a9662d8045574f0abf81c9e6bc2f180c]). Expected version 2 got 0
I20251028 02:59:52.694518 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 8ae2a510d64243a9b0fb7784ee56d29f (DEFAULT_TABLE table=default.Table_23ea93791ab84db8acf0b4e6c21141d8 [id=2fccc00ed79649eabd08b00696a27710]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.694808 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae2a510d64243a9b0fb7784ee56d29f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.695448 8562 raft_consensus.cc:493] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.695537 8562 raft_consensus.cc:515] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.695919 8562 leader_election.cc:290] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.696187 8552 ts_tablet_manager.cc:1929] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.696251 8552 log.cc:1199] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/a770dec97abb478b8c5504cc007b6092
I20251028 02:59:52.696452 8552 ts_tablet_manager.cc:1950] T a770dec97abb478b8c5504cc007b6092 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.697960 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 8ae2a510d64243a9b0fb7784ee56d29f (DEFAULT_TABLE table=default.Table_23ea93791ab84db8acf0b4e6c21141d8 [id=2fccc00ed79649eabd08b00696a27710]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.698218 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae2a510d64243a9b0fb7784ee56d29f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.698450 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 71feaa313a8f45798561741f7b5d9b20 (table default.Table_d5e26eda7deb4e4f9d3a93ac052b01c9 [id=5581f43ea3854dbeba821353ffce6ec7]). Expected version 2 got 0
I20251028 02:59:52.699728 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 8ae2a510d64243a9b0fb7784ee56d29f (DEFAULT_TABLE table=default.Table_23ea93791ab84db8acf0b4e6c21141d8 [id=2fccc00ed79649eabd08b00696a27710]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.699993 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae2a510d64243a9b0fb7784ee56d29f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.700662 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (DEFAULT_TABLE table=default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.700722 8394 tablet_service.cc:1505] Processing CreateTablet for tablet edecfb891eaa4a718ef1fdf399e7ef9d (DEFAULT_TABLE table=default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.700917 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e6d41d1cf746aab2802b5c6b2d58b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.701031 8259 tablet_service.cc:1505] Processing CreateTablet for tablet e4cdbf69eceb4bec9de3368ec959d5f0 (DEFAULT_TABLE table=default.Table_50c26dd07fa04d9a838968de2a92f0e5 [id=5290176e2166464799b4d2f4d54e97ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.701274 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4cdbf69eceb4bec9de3368ec959d5f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.702280 8119 tablet_service.cc:1505] Processing CreateTablet for tablet edecfb891eaa4a718ef1fdf399e7ef9d (DEFAULT_TABLE table=default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.702528 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edecfb891eaa4a718ef1fdf399e7ef9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.703595 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b1b505f76ef84af6bc2b78a962bf6cf9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.704649 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (DEFAULT_TABLE table=default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.704896 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e6d41d1cf746aab2802b5c6b2d58b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.704885 8546 raft_consensus.cc:359] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.704974 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edecfb891eaa4a718ef1fdf399e7ef9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.704994 8546 raft_consensus.cc:385] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.705026 8546 raft_consensus.cc:740] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.705159 8546 consensus_queue.cc:260] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.706175 8546 ts_tablet_manager.cc:1434] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.014s user 0.000s sys 0.001s
I20251028 02:59:52.706382 8546 tablet_bootstrap.cc:492] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.707054 8117 tablet_service.cc:1505] Processing CreateTablet for tablet e4cdbf69eceb4bec9de3368ec959d5f0 (DEFAULT_TABLE table=default.Table_50c26dd07fa04d9a838968de2a92f0e5 [id=5290176e2166464799b4d2f4d54e97ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.707319 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4cdbf69eceb4bec9de3368ec959d5f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.707383 8546 tablet_bootstrap.cc:654] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.708518 8259 tablet_service.cc:1505] Processing CreateTablet for tablet e7165de68f6c4987a41cd33b5c6298c4 (DEFAULT_TABLE table=default.Table_0ac5b50f10cd4c52b002d3906b21b3a7 [id=bfbe8c47585048d2a8d3d5f3d62e0c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.708729 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7165de68f6c4987a41cd33b5c6298c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.709343 8123 tablet_service.cc:1505] Processing CreateTablet for tablet e7165de68f6c4987a41cd33b5c6298c4 (DEFAULT_TABLE table=default.Table_0ac5b50f10cd4c52b002d3906b21b3a7 [id=bfbe8c47585048d2a8d3d5f3d62e0c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.709558 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7165de68f6c4987a41cd33b5c6298c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.710976 8556 raft_consensus.cc:493] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.711054 8556 raft_consensus.cc:515] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.711453 8556 leader_election.cc:290] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.711807 8400 tablet_service.cc:1505] Processing CreateTablet for tablet e4cdbf69eceb4bec9de3368ec959d5f0 (DEFAULT_TABLE table=default.Table_50c26dd07fa04d9a838968de2a92f0e5 [id=5290176e2166464799b4d2f4d54e97ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.712045 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4cdbf69eceb4bec9de3368ec959d5f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.713424 8544 tablet_bootstrap.cc:492] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.713510 8544 ts_tablet_manager.cc:1403] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20251028 02:59:52.714038 8544 raft_consensus.cc:359] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.714164 8544 raft_consensus.cc:385] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.714229 8544 raft_consensus.cc:740] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.714349 8544 consensus_queue.cc:260] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.714622 8544 ts_tablet_manager.cc:1434] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.714803 8544 tablet_bootstrap.cc:492] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.714840 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d1b772f7ad114598bf453e2265c5e436"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:52.715165 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cfd68d91fd9b4c69871383e90bd2ff85" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:52.715288 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1b772f7ad114598bf453e2265c5e436 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.715533 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 1e9234b72f4448089fe7ff6ba49483b7 (DEFAULT_TABLE table=default.Table_3d8776d793a54bd795fb38bd45442f51 [id=be1f381c3ca048d891f5d175fa72fc65]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.715729 8616 raft_consensus.cc:493] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.715747 8261 tablet_service.cc:1505] Processing CreateTablet for tablet a8331b21af69463cb7589619e784051d (DEFAULT_TABLE table=default.Table_a1cf1772f69f4e4284a15ea70a3407d6 [id=79b638b293df431cbd580e62d24e8238]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.715826 8616 raft_consensus.cc:515] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.715786 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e9234b72f4448089fe7ff6ba49483b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.716109 8263 tablet_service.cc:1505] Processing CreateTablet for tablet cf90b539dc8f47b0947d4e2ca859668a (DEFAULT_TABLE table=default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.716176 8616 leader_election.cc:290] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.716231 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b0ef051366704663b4e079ccdda63a43"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:52.716312 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf90b539dc8f47b0947d4e2ca859668a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:52.716569 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0ef051366704663b4e079ccdda63a43 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.718179 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8331b21af69463cb7589619e784051d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.718225 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 67c6ceefa3294af8b68150609f46c117 (DEFAULT_TABLE table=default.Table_12396134528e4544a80934aa91d07353 [id=86c2f59df3dd4d2195d18835dceff79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.718451 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67c6ceefa3294af8b68150609f46c117. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.720715 8544 tablet_bootstrap.cc:654] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.721876 7965 catalog_manager.cc:5649] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:52.722050 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 71feaa313a8f45798561741f7b5d9b20 (table default.Table_d5e26eda7deb4e4f9d3a93ac052b01c9 [id=5581f43ea3854dbeba821353ffce6ec7]). Expected version 2 got 0
I20251028 02:59:52.722141 7965 catalog_manager.cc:5649] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
W20251028 02:59:52.722630 8220 leader_election.cc:343] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: cfd68d91fd9b4c69871383e90bd2ff85
I20251028 02:59:52.725872 8117 tablet_service.cc:1505] Processing CreateTablet for tablet a8331b21af69463cb7589619e784051d (DEFAULT_TABLE table=default.Table_a1cf1772f69f4e4284a15ea70a3407d6 [id=79b638b293df431cbd580e62d24e8238]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.726173 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8331b21af69463cb7589619e784051d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.728905 8556 raft_consensus.cc:493] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.728892 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 1e9234b72f4448089fe7ff6ba49483b7 (DEFAULT_TABLE table=default.Table_3d8776d793a54bd795fb38bd45442f51 [id=be1f381c3ca048d891f5d175fa72fc65]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.729009 8556 raft_consensus.cc:515] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.729151 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e9234b72f4448089fe7ff6ba49483b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.729375 8556 leader_election.cc:290] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.729856 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ba114e5a94542a9b3662f28a93cd23a" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.730002 8420 raft_consensus.cc:2468] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:52.730314 8220 leader_election.cc:304] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.730430 8544 tablet_bootstrap.cc:492] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.730515 8544 ts_tablet_manager.cc:1403] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.001s
I20251028 02:59:52.730895 8556 raft_consensus.cc:2804] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.730949 8556 raft_consensus.cc:493] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.730986 8556 raft_consensus.cc:3060] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.730978 8400 tablet_service.cc:1505] Processing CreateTablet for tablet e7165de68f6c4987a41cd33b5c6298c4 (DEFAULT_TABLE table=default.Table_0ac5b50f10cd4c52b002d3906b21b3a7 [id=bfbe8c47585048d2a8d3d5f3d62e0c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.731041 8544 raft_consensus.cc:359] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.731158 8544 raft_consensus.cc:385] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.731189 8544 raft_consensus.cc:740] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.731324 8544 consensus_queue.cc:260] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.731379 8116 tablet_service.cc:1505] Processing CreateTablet for tablet cf90b539dc8f47b0947d4e2ca859668a (DEFAULT_TABLE table=default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.731602 8544 ts_tablet_manager.cc:1434] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.731613 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf90b539dc8f47b0947d4e2ca859668a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.731799 8544 tablet_bootstrap.cc:492] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.732093 8556 raft_consensus.cc:515] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.732506 8556 leader_election.cc:290] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.732808 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 5fd9cc1847ff4df7a09e2e698786d515 (DEFAULT_TABLE table=default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.733006 8544 tablet_bootstrap.cc:654] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.733294 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (DEFAULT_TABLE table=default.Table_b9407dac1631402082afb2b12dc61dbd [id=a015c31e18c74191879fe1474dcb6ab2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.733464 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f38ce803e114831ace1412abdad7126" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.733552 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff8c974231f4b8aa18c18a05ad3e738. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.733572 8285 raft_consensus.cc:2468] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:52.734762 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (DEFAULT_TABLE table=default.Table_b9407dac1631402082afb2b12dc61dbd [id=a015c31e18c74191879fe1474dcb6ab2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.735219 8262 tablet_service.cc:1505] Processing CreateTablet for tablet edecfb891eaa4a718ef1fdf399e7ef9d (DEFAULT_TABLE table=default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.735469 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edecfb891eaa4a718ef1fdf399e7ef9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.735796 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 67c6ceefa3294af8b68150609f46c117 (DEFAULT_TABLE table=default.Table_12396134528e4544a80934aa91d07353 [id=86c2f59df3dd4d2195d18835dceff79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.736023 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67c6ceefa3294af8b68150609f46c117. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.736002 8401 tablet_service.cc:1505] Processing CreateTablet for tablet cf90b539dc8f47b0947d4e2ca859668a (DEFAULT_TABLE table=default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.736045 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 1e9234b72f4448089fe7ff6ba49483b7 (DEFAULT_TABLE table=default.Table_3d8776d793a54bd795fb38bd45442f51 [id=be1f381c3ca048d891f5d175fa72fc65]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.736414 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (DEFAULT_TABLE table=default.Table_b9407dac1631402082afb2b12dc61dbd [id=a015c31e18c74191879fe1474dcb6ab2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.736764 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 67c6ceefa3294af8b68150609f46c117 (DEFAULT_TABLE table=default.Table_12396134528e4544a80934aa91d07353 [id=86c2f59df3dd4d2195d18835dceff79b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.737020 8356 leader_election.cc:304] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.737195 8616 raft_consensus.cc:2804] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.737255 8616 raft_consensus.cc:493] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.737319 8616 raft_consensus.cc:3060] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.737344 8399 tablet_service.cc:1505] Processing CreateTablet for tablet a8331b21af69463cb7589619e784051d (DEFAULT_TABLE table=default.Table_a1cf1772f69f4e4284a15ea70a3407d6 [id=79b638b293df431cbd580e62d24e8238]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.737633 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 5fd9cc1847ff4df7a09e2e698786d515 (DEFAULT_TABLE table=default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.737856 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd9cc1847ff4df7a09e2e698786d515. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.737985 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ba114e5a94542a9b3662f28a93cd23a" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.738104 8420 raft_consensus.cc:3060] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.738400 8544 tablet_bootstrap.cc:492] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.738482 8544 ts_tablet_manager.cc:1403] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:52.739006 8420 raft_consensus.cc:2468] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:52.738993 8544 raft_consensus.cc:359] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.739112 8544 raft_consensus.cc:385] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.739144 8544 raft_consensus.cc:740] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.739250 8544 consensus_queue.cc:260] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.739534 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7165de68f6c4987a41cd33b5c6298c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.739535 8544 ts_tablet_manager.cc:1434] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.739562 8220 leader_election.cc:304] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.739729 8544 tablet_bootstrap.cc:492] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.738101 8616 raft_consensus.cc:515] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.740247 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff8c974231f4b8aa18c18a05ad3e738. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.740361 8616 leader_election.cc:290] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.740561 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ba114e5a94542a9b3662f28a93cd23a" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.740566 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ba114e5a94542a9b3662f28a93cd23a" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.740800 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (DEFAULT_TABLE table=default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.741015 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e6d41d1cf746aab2802b5c6b2d58b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.733017 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd9cc1847ff4df7a09e2e698786d515. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.742022 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f38ce803e114831ace1412abdad7126" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.742399 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67c6ceefa3294af8b68150609f46c117. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:52.743661 8354 leader_election.cc:343] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.743880 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff8c974231f4b8aa18c18a05ad3e738. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.744450 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e9234b72f4448089fe7ff6ba49483b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.746347 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf90b539dc8f47b0947d4e2ca859668a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.747996 8552 tablet_replica.cc:333] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.748150 8552 raft_consensus.cc:2243] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.748255 8552 raft_consensus.cc:2272] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.749141 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6cfea1d3bb33492993e9d30e8b5af697"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.749590 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cfea1d3bb33492993e9d30e8b5af697 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.750454 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c3d6504130c54bdeade5bb23b6cd0679" } modify_external_catalogs: true
I20251028 02:59:52.750555 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c3d6504130c54bdeade5bb23b6cd0679" } modify_external_catalogs: true
W20251028 02:59:52.750689 8218 leader_election.cc:343] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.750764 8218 leader_election.cc:304] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.750813 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 6c87d03fad194ca89f1808da6dbdf0a4 from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:52.751021 8218 leader_election.cc:343] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:52.751157 8218 leader_election.cc:343] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.751291 8553 raft_consensus.cc:2804] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.751474 8553 raft_consensus.cc:697] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:52.751608 8553 consensus_queue.cc:237] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.752167 8553 raft_consensus.cc:2749] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:52.754573 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f38ce803e114831ace1412abdad7126" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.754851 8546 tablet_bootstrap.cc:492] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.754940 8546 ts_tablet_manager.cc:1403] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.049s user 0.000s sys 0.002s
I20251028 02:59:52.755563 8546 raft_consensus.cc:359] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.755674 8546 raft_consensus.cc:385] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.755707 8546 raft_consensus.cc:740] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.755801 8546 consensus_queue.cc:260] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.756081 8546 ts_tablet_manager.cc:1434] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:52.756268 8546 tablet_bootstrap.cc:492] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.757328 8546 tablet_bootstrap.cc:654] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.757460 8544 tablet_bootstrap.cc:654] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.758296 8616 raft_consensus.cc:493] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.758392 8616 raft_consensus.cc:515] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.758772 8616 leader_election.cc:290] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.759454 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0868c69e28a840b4ab9dad61d50983ec" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:52.759480 8354 leader_election.cc:343] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:52.759785 8354 leader_election.cc:343] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.760011 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6c87d03fad194ca89f1808da6dbdf0a4
I20251028 02:59:52.760288 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 6c87d03fad194ca89f1808da6dbdf0a4 deleted and replaced by tablet a3096fde2ac94ab197ce5e659c4d2be0
I20251028 02:59:52.760627 8965 raft_consensus.cc:493] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.760708 8965 raft_consensus.cc:515] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.761039 8965 leader_election.cc:290] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.761564 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f1c1e14665c4712bffa439ab98b2a6e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.761708 8420 raft_consensus.cc:2468] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:52.762131 8220 leader_election.cc:304] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.762290 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f1c1e14665c4712bffa439ab98b2a6e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.762334 8556 raft_consensus.cc:2804] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.762379 8556 raft_consensus.cc:493] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.762416 8556 raft_consensus.cc:3060] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.764356 8965 consensus_queue.cc:1048] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.765656 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f38ce803e114831ace1412abdad7126 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.767112 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 5fd9cc1847ff4df7a09e2e698786d515 (DEFAULT_TABLE table=default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.767455 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd9cc1847ff4df7a09e2e698786d515. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.769394 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b88fda3f40540f3b42ee7f33dd33251 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.770922 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8331b21af69463cb7589619e784051d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.771437 8562 raft_consensus.cc:493] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.771543 8562 raft_consensus.cc:515] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:52.771600 8218 leader_election.cc:343] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.771936 8562 leader_election.cc:290] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.771991 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet cfd68d91fd9b4c69871383e90bd2ff85 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.772619 8556 raft_consensus.cc:515] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.772774 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4726573f4b994b2591332c4a42fb10a1" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.772907 8421 raft_consensus.cc:2468] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:52.772959 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4726573f4b994b2591332c4a42fb10a1" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.773070 8285 raft_consensus.cc:2468] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:52.773331 8083 leader_election.cc:304] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:52.773329 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f1c1e14665c4712bffa439ab98b2a6e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.773361 8147 raft_consensus.cc:1275] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:52.773422 8421 raft_consensus.cc:3060] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.773763 8965 consensus_queue.cc:1048] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.774137 8556 leader_election.cc:290] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.774329 8421 raft_consensus.cc:2468] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:52.774529 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 9860023c7b564342ba3145eafe352beb (DEFAULT_TABLE table=default.Table_38f86582ea6e4e5fb73330ab8f1d4c25 [id=b072a25c1a8b4f72ba778ac32c066e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.774588 8220 leader_election.cc:304] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.774748 8556 raft_consensus.cc:2804] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.774794 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9860023c7b564342ba3145eafe352beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.775067 8556 raft_consensus.cc:697] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:52.775218 8556 consensus_queue.cc:237] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.775383 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c87d03fad194ca89f1808da6dbdf0a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a3096fde2ac94ab197ce5e659c4d2be0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.775460 8656 raft_consensus.cc:2804] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.775511 8656 raft_consensus.cc:493] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.775547 8656 raft_consensus.cc:3060] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.775890 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 7528e7ff127c475789eda805467f0b41 (DEFAULT_TABLE table=default.Table_b883c2cd0a0f49fca261f85e62e9bc18 [id=d40c0ee824ba423395e475f68276b5d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.776117 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7528e7ff127c475789eda805467f0b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.776494 8656 raft_consensus.cc:515] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.776810 8656 leader_election.cc:290] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.774529 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f1c1e14665c4712bffa439ab98b2a6e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:52.777210 8218 leader_election.cc:343] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:52.777251 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6c87d03fad194ca89f1808da6dbdf0a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6c87d03fad194ca89f1808da6dbdf0a4 already in progress: creating tablet
I20251028 02:59:52.777547 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4726573f4b994b2591332c4a42fb10a1" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:52.777647 8285 raft_consensus.cc:3060] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.777628 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4726573f4b994b2591332c4a42fb10a1" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.777729 8421 raft_consensus.cc:3060] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.778566 8285 raft_consensus.cc:2468] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:52.778566 8421 raft_consensus.cc:2468] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:52.778973 8083 leader_election.cc:304] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:52.779680 8562 raft_consensus.cc:2804] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.779805 8562 raft_consensus.cc:697] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:52.779973 8562 consensus_queue.cc:237] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.783747 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b1b505f76ef84af6bc2b78a962bf6cf9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.784150 8083 leader_election.cc:304] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:52.784386 8656 raft_consensus.cc:2749] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251028 02:59:52.784893 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 63ee5aa5a6e841fdb9244563d75394ac with delete_type TABLET_DATA_DELETED (replaced by tablet a8331b21af69463cb7589619e784051d) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.785514 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f38ce803e114831ace1412abdad7126" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:52.787492 8283 raft_consensus.cc:3060] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.788712 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 7528e7ff127c475789eda805467f0b41 (DEFAULT_TABLE table=default.Table_b883c2cd0a0f49fca261f85e62e9bc18 [id=d40c0ee824ba423395e475f68276b5d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.788802 8552 ts_tablet_manager.cc:1916] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.789002 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7528e7ff127c475789eda805467f0b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.790271 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet a770dec97abb478b8c5504cc007b6092 (table default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]) successfully deleted
I20251028 02:59:52.790575 8283 raft_consensus.cc:2468] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:52.790683 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_378281b5d5ed40cda0b93ef4a3094345" } modify_external_catalogs: true
I20251028 02:59:52.790807 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_378281b5d5ed40cda0b93ef4a3094345" } modify_external_catalogs: true
I20251028 02:59:52.790858 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 9860023c7b564342ba3145eafe352beb (DEFAULT_TABLE table=default.Table_38f86582ea6e4e5fb73330ab8f1d4c25 [id=b072a25c1a8b4f72ba778ac32c066e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.791110 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9860023c7b564342ba3145eafe352beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.791570 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 7528e7ff127c475789eda805467f0b41 (DEFAULT_TABLE table=default.Table_b883c2cd0a0f49fca261f85e62e9bc18 [id=d40c0ee824ba423395e475f68276b5d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.791796 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7528e7ff127c475789eda805467f0b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.792766 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_13cf93a2eb1f41379907b8639554e44a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.793164 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13cf93a2eb1f41379907b8639554e44a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.793790 8146 raft_consensus.cc:3060] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.793957 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c87d03fad194ca89f1808da6dbdf0a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a3096fde2ac94ab197ce5e659c4d2be0) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.794009 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 6c87d03fad194ca89f1808da6dbdf0a4 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.794174 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f38ce803e114831ace1412abdad7126 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.794248 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 9860023c7b564342ba3145eafe352beb (DEFAULT_TABLE table=default.Table_38f86582ea6e4e5fb73330ab8f1d4c25 [id=b072a25c1a8b4f72ba778ac32c066e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.794456 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9860023c7b564342ba3145eafe352beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.794473 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f0712855c8944a5bbae3fc9c069b155d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.794834 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f0712855c8944a5bbae3fc9c069b155d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.794903 8146 raft_consensus.cc:1275] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:52.795264 8553 consensus_queue.cc:1048] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.795663 8421 raft_consensus.cc:1275] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:52.795909 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 83495009a1dc4f4388d2ae65f2dda548 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.796078 8551 tablet_replica.cc:333] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.796214 8551 raft_consensus.cc:2243] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.796316 8551 raft_consensus.cc:2272] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.796708 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c87d03fad194ca89f1808da6dbdf0a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a3096fde2ac94ab197ce5e659c4d2be0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.797479 8552 ts_tablet_manager.cc:1929] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.797549 8552 log.cc:1199] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/909b08e0860b4e4f9bb1822acfd1dc02
I20251028 02:59:52.797564 8551 ts_tablet_manager.cc:1916] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.797768 8552 ts_tablet_manager.cc:1950] T 909b08e0860b4e4f9bb1822acfd1dc02 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.798415 8552 tablet_replica.cc:333] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.798475 9011 consensus_queue.cc:1048] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:52.798547 8552 raft_consensus.cc:2243] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.798637 8552 raft_consensus.cc:2272] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.798853 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_27a9af4382b34a3eab870c4d40a6c5bf" } new_table_name: "default.Table_153d49d056a3458a9565c70d954187ae" modify_external_catalogs: true
I20251028 02:59:52.799800 8551 ts_tablet_manager.cc:1929] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.799866 8551 log.cc:1199] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/6f38ce803e114831ace1412abdad7126
I20251028 02:59:52.799854 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_726e0c2d9d4044d8b79bb266e21a8d9c" } modify_external_catalogs: true
I20251028 02:59:52.799948 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_726e0c2d9d4044d8b79bb266e21a8d9c" } modify_external_catalogs: true
I20251028 02:59:52.800100 8551 ts_tablet_manager.cc:1950] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.800323 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet aec9427ed4284d66b9bd46c0e1c8774c from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.800731 8884 tablet.cc:1722] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251028 02:59:52.801275 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 909b08e0860b4e4f9bb1822acfd1dc02 (table default.Table_7fb744c8c91e4c9daadacdab1a408fa1 [id=98a07edd5f664674a2d33344038a8801]) successfully deleted
W20251028 02:59:52.801427 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 0141ebd1ddad421bbb8eca7615e78351 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0141ebd1ddad421bbb8eca7615e78351 already in progress: creating tablet
I20251028 02:59:52.805339 8356 leader_election.cc:304] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:52.805464 8356 raft_consensus.cc:2686] T 6f38ce803e114831ace1412abdad7126 P d7e4cf694f71438298dd32a346abcf75: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251028 02:59:52.806269 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0868c69e28a840b4ab9dad61d50983ec" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.806407 8283 raft_consensus.cc:2468] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:52.806818 8356 leader_election.cc:304] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.807548 8753 tablet.cc:1722] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251028 02:59:52.807869 8648 raft_consensus.cc:2804] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.807927 8648 raft_consensus.cc:493] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.807964 8648 raft_consensus.cc:3060] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.808588 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 6f38ce803e114831ace1412abdad7126 (table default.Table_828e8df9af6a48168a189935f63b2941 [id=42c89dfccc714a9eb42e3779130c8af4]) successfully deleted
W20251028 02:59:52.808804 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 6c87d03fad194ca89f1808da6dbdf0a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6c87d03fad194ca89f1808da6dbdf0a4 already in progress: creating tablet
I20251028 02:59:52.809021 8648 raft_consensus.cc:515] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.809377 8648 leader_election.cc:290] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.810324 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0868c69e28a840b4ab9dad61d50983ec" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:52.811153 8354 leader_election.cc:343] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.814744 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0868c69e28a840b4ab9dad61d50983ec" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:52.814971 8283 raft_consensus.cc:3060] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:52.815708 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet da1563d65b4d4ab48325a4f08ee89ed5 consensus state has no leader
I20251028 02:59:52.815979 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_310089f9c2b34a7e9e1c01c97fdb9b11" } new_table_name: "default.Table_c1daa144fb2045c8a183e3e7d4696f57" modify_external_catalogs: true
I20251028 02:59:52.816030 8283 raft_consensus.cc:2468] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:52.816370 8356 leader_election.cc:304] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:52.816522 8616 raft_consensus.cc:2804] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.816583 8616 raft_consensus.cc:697] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:52.816699 8616 consensus_queue.cc:237] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.817507 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_44487777ef26464c9cbbb195199247a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.817870 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_44487777ef26464c9cbbb195199247a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.818564 8656 raft_consensus.cc:493] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.818663 8656 raft_consensus.cc:515] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.819082 8656 leader_election.cc:290] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.819437 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 1ba114e5a94542a9b3662f28a93cd23a from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.819518 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3410b1b517b146d4b38cb47862c85e68" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.819654 8420 raft_consensus.cc:2393] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:52.820510 8884 tablet.cc:1722] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.822551 8753 tablet.cc:1722] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.824424 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_54bb44e4b3aa4babb80308231391809c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.824863 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_54bb44e4b3aa4babb80308231391809c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.828974 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3410b1b517b146d4b38cb47862c85e68" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.829561 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 0703e18d7dba4d628e3cc310be55fcf5 (table default.Table_8fdeffc6f6234305a829715139f150db [id=d4eaebd005bc42dd86d7125ef555546f]). Expected version 2 got 0
I20251028 02:59:52.830080 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3e3cbb0c3d914ffebfd2c976ce8b1098
I20251028 02:59:52.830447 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet edecfb891eaa4a718ef1fdf399e7ef9d from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:52.831522 8545 tablet_bootstrap.cc:492] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.831614 8545 ts_tablet_manager.cc:1403] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.141s user 0.002s sys 0.000s
I20251028 02:59:52.832383 8545 raft_consensus.cc:359] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.832491 8545 raft_consensus.cc:385] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.832525 8545 raft_consensus.cc:740] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.832677 8545 consensus_queue.cc:260] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.834859 8545 ts_tablet_manager.cc:1434] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:52.836822 8753 tablet.cc:1722] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.836874 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 002de8aede994720b8163f41938af6c4
I20251028 02:59:52.837038 8545 tablet_bootstrap.cc:492] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.837720 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 002de8aede994720b8163f41938af6c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:52.838205 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 002de8aede994720b8163f41938af6c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 002de8aede994720b8163f41938af6c4 already in progress: creating tablet
I20251028 02:59:52.838819 8545 tablet_bootstrap.cc:654] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.839672 8726 tablet.cc:1722] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251028 02:59:52.840005 8710 mvcc.cc:204] Tried to move back new op lower bound from 7215597128769761280 to 7215597127132135424. Current Snapshot: MvccSnapshot[applied={T|T < 7215597128769761280}]
I20251028 02:59:52.843430 9023 tablet.cc:1722] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.848434 8726 tablet.cc:1722] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.851176 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e3cbb0c3d914ffebfd2c976ce8b1098 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.851692 8550 tablet_replica.cc:333] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.851863 8550 raft_consensus.cc:2243] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.851967 8550 raft_consensus.cc:2272] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.852538 8550 ts_tablet_manager.cc:1916] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.854348 8550 ts_tablet_manager.cc:1929] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.854424 8550 log.cc:1199] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3e3cbb0c3d914ffebfd2c976ce8b1098
I20251028 02:59:52.854669 8550 ts_tablet_manager.cc:1950] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.855504 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3e3cbb0c3d914ffebfd2c976ce8b1098 (table default.Table_378281b5d5ed40cda0b93ef4a3094345 [id=5dd92b20a7b94b698148c30a09b3df73]) successfully deleted
I20251028 02:59:52.856988 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e3cbb0c3d914ffebfd2c976ce8b1098 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.857198 8551 tablet_replica.cc:333] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.857323 8551 raft_consensus.cc:2243] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.857424 8551 raft_consensus.cc:2272] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.857694 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 002de8aede994720b8163f41938af6c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.858175 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 83495009a1dc4f4388d2ae65f2dda548 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.858196 8551 ts_tablet_manager.cc:1916] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.858505 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d1b772f7ad114598bf453e2265c5e436" } modify_external_catalogs: true
I20251028 02:59:52.858620 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d1b772f7ad114598bf453e2265c5e436" } modify_external_catalogs: true
I20251028 02:59:52.858780 7961 catalog_manager.cc:5649] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:52.858932 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_587714a762d0451d98e0b71b42c3834c" } new_table_name: "default.Table_0e8ca626159c41aeb8e360f6e038f7a3" modify_external_catalogs: true
I20251028 02:59:52.829571 8546 tablet_bootstrap.cc:492] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.834358 8884 tablet.cc:1722] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:52.859951 7960 catalog_manager.cc:5649] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:52.860141 8546 ts_tablet_manager.cc:1403] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.104s user 0.000s sys 0.002s
I20251028 02:59:52.857263 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 3e3cbb0c3d914ffebfd2c976ce8b1098 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.857668 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 002de8aede994720b8163f41938af6c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.860699 8546 raft_consensus.cc:359] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.860805 8546 raft_consensus.cc:385] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.860834 8546 raft_consensus.cc:740] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.860956 8546 consensus_queue.cc:260] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.861253 8546 ts_tablet_manager.cc:1434] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.861457 8546 tablet_bootstrap.cc:492] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.861615 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_97b3a76d004f47c981178cf580a95985" } modify_external_catalogs: true
I20251028 02:59:52.861712 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_97b3a76d004f47c981178cf580a95985" } modify_external_catalogs: true
I20251028 02:59:52.862663 8546 tablet_bootstrap.cc:654] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.865995 8083 leader_election.cc:304] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:52.866250 8656 raft_consensus.cc:3060] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.867179 8122 tablet_service.cc:1505] Processing CreateTablet for tablet d107a510c8664999901320581e885ba9 (DEFAULT_TABLE table=default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.867278 8656 raft_consensus.cc:2749] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:52.867574 8116 tablet_service.cc:1505] Processing CreateTablet for tablet c083ca984a7347a6b5bb0f3a949a5f75 (DEFAULT_TABLE table=default.Table_d1b772f7ad114598bf453e2265c5e436 [id=73579e6e9da94fa699b3cb5bb5cec89e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.867992 8551 ts_tablet_manager.cc:1929] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.868069 8551 log.cc:1199] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3e3cbb0c3d914ffebfd2c976ce8b1098
I20251028 02:59:52.868332 8551 ts_tablet_manager.cc:1950] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.867628 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d107a510c8664999901320581e885ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.869858 8117 tablet_service.cc:1505] Processing CreateTablet for tablet a3096fde2ac94ab197ce5e659c4d2be0 (DEFAULT_TABLE table=default.Table_bcdd1e4fff1a45e39b03d0c7f2268672 [id=a6d84958fd654087940c9806e2e3fbc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.869993 8552 ts_tablet_manager.cc:1916] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.870069 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3096fde2ac94ab197ce5e659c4d2be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.871608 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c083ca984a7347a6b5bb0f3a949a5f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.872231 8552 ts_tablet_manager.cc:1929] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.872294 8552 log.cc:1199] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/6f38ce803e114831ace1412abdad7126
I20251028 02:59:52.872493 8552 ts_tablet_manager.cc:1950] T 6f38ce803e114831ace1412abdad7126 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.872610 8119 tablet_service.cc:1505] Processing CreateTablet for tablet ba35c8fe21f140cda8a4b1ad7568ec75 (DEFAULT_TABLE table=default.Table_2ad18e95c3a14e039a63bebaa46268fa [id=abb728d9aa374dce81ffa90585353fa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.872840 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba35c8fe21f140cda8a4b1ad7568ec75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.873059 8552 tablet_replica.cc:333] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.873175 8552 raft_consensus.cc:2243] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.873265 8552 raft_consensus.cc:2272] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.874078 8544 tablet_bootstrap.cc:492] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.874164 8544 ts_tablet_manager.cc:1403] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.134s user 0.002s sys 0.000s
I20251028 02:59:52.874697 8544 raft_consensus.cc:359] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.874806 8544 raft_consensus.cc:385] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.874845 8546 tablet_bootstrap.cc:492] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.874863 8544 raft_consensus.cc:740] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.874925 8546 ts_tablet_manager.cc:1403] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20251028 02:59:52.874985 8544 consensus_queue.cc:260] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.875283 8544 ts_tablet_manager.cc:1434] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.875416 8546 raft_consensus.cc:359] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.875488 8544 tablet_bootstrap.cc:492] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.875514 8546 raft_consensus.cc:385] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.875546 8546 raft_consensus.cc:740] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.875658 8546 consensus_queue.cc:260] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.875983 8546 ts_tablet_manager.cc:1434] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.876205 8546 tablet_bootstrap.cc:492] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.876349 8551 tablet_replica.cc:333] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.876487 8551 raft_consensus.cc:2243] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.876588 8551 raft_consensus.cc:2272] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.876617 8544 tablet_bootstrap.cc:654] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.877276 8546 tablet_bootstrap.cc:654] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.877753 8552 ts_tablet_manager.cc:1916] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.879072 8551 ts_tablet_manager.cc:1916] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.879685 8546 tablet_bootstrap.cc:492] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.879763 8546 ts_tablet_manager.cc:1403] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:52.880026 8544 tablet_bootstrap.cc:492] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.880121 8544 ts_tablet_manager.cc:1403] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20251028 02:59:52.880272 8546 raft_consensus.cc:359] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.880371 8546 raft_consensus.cc:385] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.880404 8546 raft_consensus.cc:740] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.880515 8546 consensus_queue.cc:260] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.880556 8544 raft_consensus.cc:359] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.880641 8544 raft_consensus.cc:385] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.880678 8544 raft_consensus.cc:740] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.880846 8546 ts_tablet_manager.cc:1434] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:52.880803 8544 consensus_queue.cc:260] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.881023 8546 tablet_bootstrap.cc:492] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.881109 8544 ts_tablet_manager.cc:1434] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:52.881273 8544 tablet_bootstrap.cc:492] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.882273 8544 tablet_bootstrap.cc:654] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.882030 8546 tablet_bootstrap.cc:654] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.887655 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 8cb97d5192684d748b8b485f891b0dc0 (table default.Table_8bd9c3af53644cbcb68461ae9433a9f1 [id=a9662d8045574f0abf81c9e6bc2f180c]). Expected version 2 got 0
I20251028 02:59:52.887964 8552 ts_tablet_manager.cc:1929] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.888031 8552 log.cc:1199] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/8b88fda3f40540f3b42ee7f33dd33251
I20251028 02:59:52.888239 8552 ts_tablet_manager.cc:1950] T 8b88fda3f40540f3b42ee7f33dd33251 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.888306 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet aec9427ed4284d66b9bd46c0e1c8774c
I20251028 02:59:52.888573 7963 catalog_manager.cc:6719] ReplaceTablet: tablet aec9427ed4284d66b9bd46c0e1c8774c deleted and replaced by tablet de38b2d54cec4f4e892047f4650f611b
I20251028 02:59:52.888871 8552 tablet_replica.cc:333] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.889000 8552 raft_consensus.cc:2243] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.889102 8552 raft_consensus.cc:2272] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.889883 8544 tablet_bootstrap.cc:492] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.889963 8544 ts_tablet_manager.cc:1403] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20251028 02:59:52.890406 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet aec9427ed4284d66b9bd46c0e1c8774c with delete_type TABLET_DATA_DELETED (replaced by tablet de38b2d54cec4f4e892047f4650f611b) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.890403 8544 raft_consensus.cc:359] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.890568 8544 raft_consensus.cc:385] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.890594 8544 raft_consensus.cc:740] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.890698 8544 consensus_queue.cc:260] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.890949 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 56c8969da9dc464d9b463205e3a27965 (DEFAULT_TABLE table=default.Table_6cfea1d3bb33492993e9d30e8b5af697 [id=091f678d62af4a0792dfccedb3db996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.891058 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet e359f0121c334cc1a8b3eedbbe64bda2
I20251028 02:59:52.891222 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c8969da9dc464d9b463205e3a27965. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.891475 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c083ca984a7347a6b5bb0f3a949a5f75
I20251028 02:59:52.890962 8544 ts_tablet_manager.cc:1434] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.891810 8544 tablet_bootstrap.cc:492] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 02:59:52.891988 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7f804f81d70749b5bdefeca4e0bad6fe consensus state has no leader
I20251028 02:59:52.892753 8544 tablet_bootstrap.cc:654] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.893399 7963 catalog_manager.cc:5649] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:52.893983 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 025a4d3e71ff443c8bdf649055a17d54
I20251028 02:59:52.894371 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet edecfb891eaa4a718ef1fdf399e7ef9d
I20251028 02:59:52.894608 7966 catalog_manager.cc:6719] ReplaceTablet: tablet edecfb891eaa4a718ef1fdf399e7ef9d deleted and replaced by tablet 7801d57550dc40c39f2d3cf7e11e3713
I20251028 02:59:52.897229 8546 tablet_bootstrap.cc:492] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.897318 8546 ts_tablet_manager.cc:1403] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20251028 02:59:52.897876 8546 raft_consensus.cc:359] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.897984 8546 raft_consensus.cc:385] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.898010 8546 raft_consensus.cc:740] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.898149 8546 consensus_queue.cc:260] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:52.898263 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet aec9427ed4284d66b9bd46c0e1c8774c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aec9427ed4284d66b9bd46c0e1c8774c already in progress: creating tablet
I20251028 02:59:52.898450 8546 ts_tablet_manager.cc:1434] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.898664 8546 tablet_bootstrap.cc:492] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.899693 8546 tablet_bootstrap.cc:654] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.900734 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet edecfb891eaa4a718ef1fdf399e7ef9d with delete_type TABLET_DATA_DELETED (replaced by tablet 7801d57550dc40c39f2d3cf7e11e3713) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.900784 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 025a4d3e71ff443c8bdf649055a17d54 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.900940 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet c083ca984a7347a6b5bb0f3a949a5f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.901201 8550 tablet_replica.cc:333] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.901398 8544 tablet_bootstrap.cc:492] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.901479 8544 ts_tablet_manager.cc:1403] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20251028 02:59:52.901924 8544 raft_consensus.cc:359] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.902014 8544 raft_consensus.cc:385] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.902042 8544 raft_consensus.cc:740] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.902164 8544 consensus_queue.cc:260] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.901340 8550 raft_consensus.cc:2243] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.902436 8544 ts_tablet_manager.cc:1434] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.902500 8550 raft_consensus.cc:2272] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.902629 8544 tablet_bootstrap.cc:492] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.903100 8550 ts_tablet_manager.cc:1916] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.903687 8544 tablet_bootstrap.cc:654] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:52.904086 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet edecfb891eaa4a718ef1fdf399e7ef9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edecfb891eaa4a718ef1fdf399e7ef9d already in progress: creating tablet
I20251028 02:59:52.906103 8550 ts_tablet_manager.cc:1929] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:52.906168 8550 log.cc:1199] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/025a4d3e71ff443c8bdf649055a17d54
I20251028 02:59:52.907063 8550 ts_tablet_manager.cc:1950] T 025a4d3e71ff443c8bdf649055a17d54 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.907732 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 025a4d3e71ff443c8bdf649055a17d54 (table default.Table_97b3a76d004f47c981178cf580a95985 [id=7d237f6c141e4d5e821305500a020dc0]) successfully deleted
W20251028 02:59:52.908200 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet c083ca984a7347a6b5bb0f3a949a5f75 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c083ca984a7347a6b5bb0f3a949a5f75 already in progress: creating tablet
I20251028 02:59:52.908763 8545 tablet_bootstrap.cc:492] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.908869 8545 ts_tablet_manager.cc:1403] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.072s user 0.002s sys 0.000s
I20251028 02:59:52.910727 8552 ts_tablet_manager.cc:1916] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.912240 8551 ts_tablet_manager.cc:1929] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.913309 8552 ts_tablet_manager.cc:1929] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.913388 8552 log.cc:1199] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/cfd68d91fd9b4c69871383e90bd2ff85
I20251028 02:59:52.913390 8551 log.cc:1199] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/83495009a1dc4f4388d2ae65f2dda548
I20251028 02:59:52.913589 8552 ts_tablet_manager.cc:1950] T cfd68d91fd9b4c69871383e90bd2ff85 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.913671 8551 ts_tablet_manager.cc:1950] T 83495009a1dc4f4388d2ae65f2dda548 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.914408 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1ba114e5a94542a9b3662f28a93cd23a
I20251028 02:59:52.914511 8545 raft_consensus.cc:359] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.914613 8545 raft_consensus.cc:385] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.914645 8545 raft_consensus.cc:740] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.914708 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 1ba114e5a94542a9b3662f28a93cd23a deleted and replaced by tablet 69f1f6895a604b858b28a3e0241245de
I20251028 02:59:52.914793 8552 tablet_replica.cc:333] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.914849 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet e359f0121c334cc1a8b3eedbbe64bda2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.914918 8552 raft_consensus.cc:2243] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.915009 8552 raft_consensus.cc:2272] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.915036 8545 consensus_queue.cc:260] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.915490 8552 ts_tablet_manager.cc:1916] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.917958 8552 ts_tablet_manager.cc:1929] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.918033 8552 log.cc:1199] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/83495009a1dc4f4388d2ae65f2dda548
I20251028 02:59:52.918239 8552 ts_tablet_manager.cc:1950] T 83495009a1dc4f4388d2ae65f2dda548 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.918896 8545 ts_tablet_manager.cc:1434] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:52.918988 8552 tablet_replica.cc:333] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.918466 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 1ba114e5a94542a9b3662f28a93cd23a with delete_type TABLET_DATA_DELETED (replaced by tablet 69f1f6895a604b858b28a3e0241245de) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.919124 8552 raft_consensus.cc:2243] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:52.919291 8552 raft_consensus.cc:2272] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.919760 8552 ts_tablet_manager.cc:1916] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.921636 8545 tablet_bootstrap.cc:492] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.921717 8552 ts_tablet_manager.cc:1929] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.921778 8552 log.cc:1199] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3e3cbb0c3d914ffebfd2c976ce8b1098
I20251028 02:59:52.922014 8552 ts_tablet_manager.cc:1950] T 3e3cbb0c3d914ffebfd2c976ce8b1098 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.923084 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 56c8969da9dc464d9b463205e3a27965 (DEFAULT_TABLE table=default.Table_6cfea1d3bb33492993e9d30e8b5af697 [id=091f678d62af4a0792dfccedb3db996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.923384 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c8969da9dc464d9b463205e3a27965. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.926120 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 56c8969da9dc464d9b463205e3a27965 (DEFAULT_TABLE table=default.Table_6cfea1d3bb33492993e9d30e8b5af697 [id=091f678d62af4a0792dfccedb3db996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.926151 8394 tablet_service.cc:1505] Processing CreateTablet for tablet c083ca984a7347a6b5bb0f3a949a5f75 (DEFAULT_TABLE table=default.Table_d1b772f7ad114598bf453e2265c5e436 [id=73579e6e9da94fa699b3cb5bb5cec89e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.926385 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c083ca984a7347a6b5bb0f3a949a5f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.926385 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c8969da9dc464d9b463205e3a27965. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.926766 8260 tablet_service.cc:1505] Processing CreateTablet for tablet d107a510c8664999901320581e885ba9 (DEFAULT_TABLE table=default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.926966 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d107a510c8664999901320581e885ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.927335 8262 tablet_service.cc:1505] Processing CreateTablet for tablet ba35c8fe21f140cda8a4b1ad7568ec75 (DEFAULT_TABLE table=default.Table_2ad18e95c3a14e039a63bebaa46268fa [id=abb728d9aa374dce81ffa90585353fa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.927495 8261 tablet_service.cc:1505] Processing CreateTablet for tablet c083ca984a7347a6b5bb0f3a949a5f75 (DEFAULT_TABLE table=default.Table_d1b772f7ad114598bf453e2265c5e436 [id=73579e6e9da94fa699b3cb5bb5cec89e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.927618 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba35c8fe21f140cda8a4b1ad7568ec75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.927886 8400 tablet_service.cc:1505] Processing CreateTablet for tablet d107a510c8664999901320581e885ba9 (DEFAULT_TABLE table=default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.928220 8399 tablet_service.cc:1505] Processing CreateTablet for tablet ba35c8fe21f140cda8a4b1ad7568ec75 (DEFAULT_TABLE table=default.Table_2ad18e95c3a14e039a63bebaa46268fa [id=abb728d9aa374dce81ffa90585353fa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.928416 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c083ca984a7347a6b5bb0f3a949a5f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.930363 8262 tablet_service.cc:1505] Processing CreateTablet for tablet a3096fde2ac94ab197ce5e659c4d2be0 (DEFAULT_TABLE table=default.Table_bcdd1e4fff1a45e39b03d0c7f2268672 [id=a6d84958fd654087940c9806e2e3fbc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.930434 8401 tablet_service.cc:1505] Processing CreateTablet for tablet a3096fde2ac94ab197ce5e659c4d2be0 (DEFAULT_TABLE table=default.Table_bcdd1e4fff1a45e39b03d0c7f2268672 [id=a6d84958fd654087940c9806e2e3fbc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.930588 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3096fde2ac94ab197ce5e659c4d2be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.930626 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba35c8fe21f140cda8a4b1ad7568ec75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.930895 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d107a510c8664999901320581e885ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.931173 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3096fde2ac94ab197ce5e659c4d2be0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.933502 8545 tablet_bootstrap.cc:654] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.934809 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet aec9427ed4284d66b9bd46c0e1c8774c with delete_type TABLET_DATA_DELETED (replaced by tablet de38b2d54cec4f4e892047f4650f611b) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.935554 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet e359f0121c334cc1a8b3eedbbe64bda2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.936190 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet c083ca984a7347a6b5bb0f3a949a5f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:52.936163 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e359f0121c334cc1a8b3eedbbe64bda2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e359f0121c334cc1a8b3eedbbe64bda2 already in progress: creating tablet
I20251028 02:59:52.936236 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet c083ca984a7347a6b5bb0f3a949a5f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:52.936347 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1ba114e5a94542a9b3662f28a93cd23a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ba114e5a94542a9b3662f28a93cd23a already in progress: creating tablet
I20251028 02:59:52.937165 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 025a4d3e71ff443c8bdf649055a17d54 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.937182 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 025a4d3e71ff443c8bdf649055a17d54 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.937315 8551 tablet_replica.cc:333] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.937333 8552 tablet_replica.cc:333] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.937366 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet aec9427ed4284d66b9bd46c0e1c8774c with delete_type TABLET_DATA_DELETED (replaced by tablet de38b2d54cec4f4e892047f4650f611b) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.937450 8551 raft_consensus.cc:2243] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:52.937477 8552 raft_consensus.cc:2243] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.937456 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet e359f0121c334cc1a8b3eedbbe64bda2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.937667 8551 raft_consensus.cc:2272] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.937695 8552 raft_consensus.cc:2272] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.938235 8545 tablet_bootstrap.cc:492] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:52.938262 8551 ts_tablet_manager.cc:1916] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.938320 8545 ts_tablet_manager.cc:1403] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 02:59:52.938482 8552 ts_tablet_manager.cc:1916] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.938817 8545 raft_consensus.cc:359] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.938930 8545 raft_consensus.cc:385] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.938963 8545 raft_consensus.cc:740] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.939414 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet edecfb891eaa4a718ef1fdf399e7ef9d with delete_type TABLET_DATA_DELETED (replaced by tablet 7801d57550dc40c39f2d3cf7e11e3713) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.939659 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet edecfb891eaa4a718ef1fdf399e7ef9d with delete_type TABLET_DATA_DELETED (replaced by tablet 7801d57550dc40c39f2d3cf7e11e3713) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.939790 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 1ba114e5a94542a9b3662f28a93cd23a with delete_type TABLET_DATA_DELETED (replaced by tablet 69f1f6895a604b858b28a3e0241245de) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:52.939975 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 1ba114e5a94542a9b3662f28a93cd23a with delete_type TABLET_DATA_DELETED (replaced by tablet 69f1f6895a604b858b28a3e0241245de) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:52.940460 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3e3cbb0c3d914ffebfd2c976ce8b1098 (table default.Table_378281b5d5ed40cda0b93ef4a3094345 [id=5dd92b20a7b94b698148c30a09b3df73]) successfully deleted
W20251028 02:59:52.940634 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 002de8aede994720b8163f41938af6c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 002de8aede994720b8163f41938af6c4 already in progress: creating tablet
I20251028 02:59:52.940735 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 83495009a1dc4f4388d2ae65f2dda548 (table default.Table_309f970a07c6425bb6e764dfb593bc80 [id=27a892441799474b9572251752e147f3]) successfully deleted
W20251028 02:59:52.941151 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet c083ca984a7347a6b5bb0f3a949a5f75 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c083ca984a7347a6b5bb0f3a949a5f75 already in progress: creating tablet
I20251028 02:59:52.941596 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 6f38ce803e114831ace1412abdad7126 (table default.Table_828e8df9af6a48168a189935f63b2941 [id=42c89dfccc714a9eb42e3779130c8af4]) successfully deleted
I20251028 02:59:52.941614 8552 ts_tablet_manager.cc:1929] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:52.941679 8552 log.cc:1199] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/025a4d3e71ff443c8bdf649055a17d54
I20251028 02:59:52.941679 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 8b88fda3f40540f3b42ee7f33dd33251 (table default.Table_7fb744c8c91e4c9daadacdab1a408fa1 [id=98a07edd5f664674a2d33344038a8801]) successfully deleted
I20251028 02:59:52.941746 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet cfd68d91fd9b4c69871383e90bd2ff85 (table default.Table_620185a247324af6b2054263457ec7da [id=c33a0072d1384953b73c2e3d90176cc4]) successfully deleted
W20251028 02:59:52.941839 7955 catalog_manager.cc:4952] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 63ee5aa5a6e841fdb9244563d75394ac because the tablet was not found. No further retry: Not found: Tablet not found: 63ee5aa5a6e841fdb9244563d75394ac
W20251028 02:59:52.941939 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 6c87d03fad194ca89f1808da6dbdf0a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6c87d03fad194ca89f1808da6dbdf0a4 already in progress: creating tablet
I20251028 02:59:52.941948 8552 ts_tablet_manager.cc:1950] T 025a4d3e71ff443c8bdf649055a17d54 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.941957 8551 ts_tablet_manager.cc:1929] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:52.942013 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 83495009a1dc4f4388d2ae65f2dda548 (table default.Table_309f970a07c6425bb6e764dfb593bc80 [id=27a892441799474b9572251752e147f3]) successfully deleted
I20251028 02:59:52.942016 8551 log.cc:1199] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/025a4d3e71ff443c8bdf649055a17d54
I20251028 02:59:52.942262 8551 ts_tablet_manager.cc:1950] T 025a4d3e71ff443c8bdf649055a17d54 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.942488 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4f4721dca71f432b82e320f8fa9b0cf5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:52.942662 8552 tablet_replica.cc:333] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:52.942812 8552 raft_consensus.cc:2243] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
W20251028 02:59:52.942904 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f4721dca71f432b82e320f8fa9b0cf5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.942960 8552 pending_rounds.cc:70] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:52.942992 8552 pending_rounds.cc:77] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597128711958528 op_type: NO_OP noop_request { }
I20251028 02:59:52.943045 8552 raft_consensus.cc:2889] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:52.943096 8552 raft_consensus.cc:2272] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.943333 8551 tablet_replica.cc:333] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:52.943460 8551 raft_consensus.cc:2243] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.943538 8551 raft_consensus.cc:2272] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.943581 8545 consensus_queue.cc:260] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.943805 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c1daa144fb2045c8a183e3e7d4696f57" } new_table_name: "default.Table_a18c2020018b46ad99fea42906a9e93d" modify_external_catalogs: true
I20251028 02:59:52.944033 8551 ts_tablet_manager.cc:1916] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.944584 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6cfea1d3bb33492993e9d30e8b5af697" } new_table_name: "default.Table_46d5cdd6882c46da97c21a768a91d858" modify_external_catalogs: true
I20251028 02:59:52.946161 8551 ts_tablet_manager.cc:1929] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.946220 8551 log.cc:1199] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/1ba114e5a94542a9b3662f28a93cd23a
I20251028 02:59:52.946475 8545 ts_tablet_manager.cc:1434] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20251028 02:59:52.946944 8551 ts_tablet_manager.cc:1950] T 1ba114e5a94542a9b3662f28a93cd23a P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:52.946935 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_cc38c53f9a064f3f8af6b4d688c1aada"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.947330 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc38c53f9a064f3f8af6b4d688c1aada in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.948099 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f2b0692abe27469a8f0728978e62ae26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:52.948419 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2b0692abe27469a8f0728978e62ae26 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:52.949193 8545 tablet_bootstrap.cc:492] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:52.950330 8545 tablet_bootstrap.cc:654] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.942081 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3e3cbb0c3d914ffebfd2c976ce8b1098 (table default.Table_378281b5d5ed40cda0b93ef4a3094345 [id=5dd92b20a7b94b698148c30a09b3df73]) successfully deleted
W20251028 02:59:52.951681 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 002de8aede994720b8163f41938af6c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 002de8aede994720b8163f41938af6c4 already in progress: creating tablet
I20251028 02:59:52.951732 8546 tablet_bootstrap.cc:492] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.951819 8546 ts_tablet_manager.cc:1403] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.000s
W20251028 02:59:52.952010 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet aec9427ed4284d66b9bd46c0e1c8774c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aec9427ed4284d66b9bd46c0e1c8774c already in progress: creating tablet
W20251028 02:59:52.952121 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet e359f0121c334cc1a8b3eedbbe64bda2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e359f0121c334cc1a8b3eedbbe64bda2 already in progress: creating tablet
W20251028 02:59:52.952203 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet c083ca984a7347a6b5bb0f3a949a5f75 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c083ca984a7347a6b5bb0f3a949a5f75 already in progress: creating tablet
I20251028 02:59:52.952319 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 025a4d3e71ff443c8bdf649055a17d54 (table default.Table_97b3a76d004f47c981178cf580a95985 [id=7d237f6c141e4d5e821305500a020dc0]) successfully deleted
I20251028 02:59:52.952297 8546 raft_consensus.cc:359] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.952395 8546 raft_consensus.cc:385] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.952425 8546 raft_consensus.cc:740] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
W20251028 02:59:52.952435 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet edecfb891eaa4a718ef1fdf399e7ef9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edecfb891eaa4a718ef1fdf399e7ef9d already in progress: creating tablet
I20251028 02:59:52.952539 8546 consensus_queue.cc:260] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.952806 8546 ts_tablet_manager.cc:1434] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:52.953018 8546 tablet_bootstrap.cc:492] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.953244 8544 tablet_bootstrap.cc:492] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.953341 8544 ts_tablet_manager.cc:1403] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.051s user 0.000s sys 0.002s
I20251028 02:59:52.953867 8544 raft_consensus.cc:359] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.953975 8544 raft_consensus.cc:385] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.954005 8544 raft_consensus.cc:740] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.954043 8546 tablet_bootstrap.cc:654] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.954130 8544 consensus_queue.cc:260] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.954416 8544 ts_tablet_manager.cc:1434] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.954658 8544 tablet_bootstrap.cc:492] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:52.955047 8552 ts_tablet_manager.cc:1916] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.955708 8544 tablet_bootstrap.cc:654] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.956518 8965 raft_consensus.cc:493] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.956594 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 0703e18d7dba4d628e3cc310be55fcf5 (table default.Table_8fdeffc6f6234305a829715139f150db [id=d4eaebd005bc42dd86d7125ef555546f]). Expected version 2 got 0
I20251028 02:59:52.956598 8965 raft_consensus.cc:515] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.956791 7966 catalog_manager.cc:5649] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:52.957016 8965 leader_election.cc:290] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.957111 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_7778bd93ae56485f921a20d0df78a3ec [id=7a2a6ab3c9744942a9413c6cb057f527] alter complete (version 4)
I20251028 02:59:52.957471 8552 ts_tablet_manager.cc:1929] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:52.957532 8552 log.cc:1199] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/1ba114e5a94542a9b3662f28a93cd23a
I20251028 02:59:52.957477 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488bc52f1219478ab8843c9b1d79de2b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.957633 8420 raft_consensus.cc:2468] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:52.957643 7963 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_8bd9c3af53644cbcb68461ae9433a9f1 [id=a9662d8045574f0abf81c9e6bc2f180c] alter complete (version 2)
I20251028 02:59:52.957770 8552 ts_tablet_manager.cc:1950] T 1ba114e5a94542a9b3662f28a93cd23a P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:52.958173 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488bc52f1219478ab8843c9b1d79de2b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.958302 8220 leader_election.cc:304] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
W20251028 02:59:52.958613 8218 leader_election.cc:343] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.958923 8556 raft_consensus.cc:2804] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.958976 8556 raft_consensus.cc:493] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.959013 8556 raft_consensus.cc:3060] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.959930 8556 raft_consensus.cc:515] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.960291 8556 leader_election.cc:290] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.960366 8556 raft_consensus.cc:493] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.960414 8556 raft_consensus.cc:515] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.960723 8556 leader_election.cc:290] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.961550 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0703e18d7dba4d628e3cc310be55fcf5" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.961680 8421 raft_consensus.cc:2468] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:52.961882 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488bc52f1219478ab8843c9b1d79de2b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.961967 8420 raft_consensus.cc:3060] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.962177 8546 tablet_bootstrap.cc:492] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.962255 8546 ts_tablet_manager.cc:1403] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 02:59:52.962735 8546 raft_consensus.cc:359] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.962833 8546 raft_consensus.cc:385] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.962862 8546 raft_consensus.cc:740] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.962957 8546 consensus_queue.cc:260] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.963037 8420 raft_consensus.cc:2468] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:52.963208 8546 ts_tablet_manager.cc:1434] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:52.963424 8546 tablet_bootstrap.cc:492] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.963676 8116 tablet_service.cc:1505] Processing CreateTablet for tablet cdd92cd8a5194e22a91e59aaa4793135 (DEFAULT_TABLE table=default.Table_54bb44e4b3aa4babb80308231391809c [id=2bd44c4bfd2741c7bec6f973ffc7682d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.963963 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd92cd8a5194e22a91e59aaa4793135. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.964427 8546 tablet_bootstrap.cc:654] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:52.967093 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488bc52f1219478ab8843c9b1d79de2b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.967149 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0703e18d7dba4d628e3cc310be55fcf5" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:52.967381 8546 tablet_bootstrap.cc:492] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.967454 8546 ts_tablet_manager.cc:1403] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20251028 02:59:52.967535 8220 leader_election.cc:304] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.967715 8220 leader_election.cc:304] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.967870 8546 raft_consensus.cc:359] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.967940 9011 raft_consensus.cc:2804] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:52.968006 9011 raft_consensus.cc:493] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:52.968058 9011 raft_consensus.cc:3060] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.969082 9011 raft_consensus.cc:515] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.969451 9011 leader_election.cc:290] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:52.969547 9011 raft_consensus.cc:2804] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.969605 9011 raft_consensus.cc:697] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:52.969736 9011 consensus_queue.cc:237] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.969820 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0703e18d7dba4d628e3cc310be55fcf5" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:52.969931 8420 raft_consensus.cc:3060] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.970868 8420 raft_consensus.cc:2468] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:52.971474 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 2ab414dbe9144aef8fd72a7f5a95577d (DEFAULT_TABLE table=default.Table_44487777ef26464c9cbbb195199247a0 [id=f04d44307e3c4c07a2403477d11e9fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.971755 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ab414dbe9144aef8fd72a7f5a95577d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.972465 8220 leader_election.cc:304] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:52.972630 8553 raft_consensus.cc:2804] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:52.972693 8553 raft_consensus.cc:697] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:52.972805 8553 consensus_queue.cc:237] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:52.974251 8218 leader_election.cc:343] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:52.974423 8218 leader_election.cc:343] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.974774 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0703e18d7dba4d628e3cc310be55fcf5" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:52.974903 8119 tablet_service.cc:1505] Processing CreateTablet for tablet e6f9b707ec0345249c3efec1c24a956e (DEFAULT_TABLE table=default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.975131 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6f9b707ec0345249c3efec1c24a956e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.967957 8546 raft_consensus.cc:385] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.978925 8546 raft_consensus.cc:740] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.979133 8546 consensus_queue.cc:260] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.979537 8546 ts_tablet_manager.cc:1434] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20251028 02:59:52.979789 8546 tablet_bootstrap.cc:492] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.980924 8546 tablet_bootstrap.cc:654] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:52.981889 8218 leader_election.cc:343] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:52.985751 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet cfd68d91fd9b4c69871383e90bd2ff85 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:52.986032 8550 tablet_replica.cc:333] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:52.986187 8550 raft_consensus.cc:2243] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:52.986289 8550 raft_consensus.cc:2272] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:52.986423 8546 tablet_bootstrap.cc:492] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:52.986519 8546 ts_tablet_manager.cc:1403] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:52.986773 8550 ts_tablet_manager.cc:1916] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:52.987035 8546 raft_consensus.cc:359] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.987138 8546 raft_consensus.cc:385] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.987174 8546 raft_consensus.cc:740] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.987509 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 1094f4d2872449599e3eaf286def4f18 (DEFAULT_TABLE table=default.Table_f0712855c8944a5bbae3fc9c069b155d [id=c22cf53127924afcbc55ea4a1fd8c04f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:52.987775 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1094f4d2872449599e3eaf286def4f18. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:52.989148 8550 ts_tablet_manager.cc:1929] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:52.989210 8550 log.cc:1199] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/cfd68d91fd9b4c69871383e90bd2ff85
I20251028 02:59:52.989408 8550 ts_tablet_manager.cc:1950] T cfd68d91fd9b4c69871383e90bd2ff85 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:52.989795 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet cfd68d91fd9b4c69871383e90bd2ff85 (table default.Table_620185a247324af6b2054263457ec7da [id=c33a0072d1384953b73c2e3d90176cc4]) successfully deleted
I20251028 02:59:52.990031 8546 consensus_queue.cc:260] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.990404 8546 ts_tablet_manager.cc:1434] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:52.994438 8656 raft_consensus.cc:493] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:52.994553 8656 raft_consensus.cc:515] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:52.994954 8656 leader_election.cc:290] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:52.995579 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0868c69e28a840b4ab9dad61d50983ec" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:52.995642 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0868c69e28a840b4ab9dad61d50983ec" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:52.995783 8283 raft_consensus.cc:2393] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 02:59:52.996084 8546 tablet_bootstrap.cc:492] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:52.996160 8544 tablet_bootstrap.cc:492] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:52.996244 8544 ts_tablet_manager.cc:1403] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.002s
I20251028 02:59:52.996445 8083 leader_election.cc:304] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:52.996604 8656 raft_consensus.cc:3060] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:52.996738 8544 raft_consensus.cc:359] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.996837 8544 raft_consensus.cc:385] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:52.996872 8544 raft_consensus.cc:740] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:52.997013 8544 consensus_queue.cc:260] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:52.997306 8544 ts_tablet_manager.cc:1434] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:52.997601 8656 raft_consensus.cc:2749] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:52.998414 8546 tablet_bootstrap.cc:654] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:52.998757 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 56c8969da9dc464d9b463205e3a27965 consensus state has no leader
I20251028 02:59:52.998997 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 025a4d3e71ff443c8bdf649055a17d54 (table default.Table_97b3a76d004f47c981178cf580a95985 [id=7d237f6c141e4d5e821305500a020dc0]) successfully deleted
W20251028 02:59:52.999176 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet aec9427ed4284d66b9bd46c0e1c8774c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aec9427ed4284d66b9bd46c0e1c8774c already in progress: creating tablet
W20251028 02:59:52.999274 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet e359f0121c334cc1a8b3eedbbe64bda2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e359f0121c334cc1a8b3eedbbe64bda2 already in progress: creating tablet
W20251028 02:59:52.999387 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet edecfb891eaa4a718ef1fdf399e7ef9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edecfb891eaa4a718ef1fdf399e7ef9d already in progress: creating tablet
I20251028 02:59:52.999466 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 1ba114e5a94542a9b3662f28a93cd23a (table default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]) successfully deleted
I20251028 02:59:53.000267 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f3ea294303c54da8845388204f1b3a5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.000408 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 1094f4d2872449599e3eaf286def4f18 (DEFAULT_TABLE table=default.Table_f0712855c8944a5bbae3fc9c069b155d [id=c22cf53127924afcbc55ea4a1fd8c04f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.000679 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1094f4d2872449599e3eaf286def4f18. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:53.000663 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3ea294303c54da8845388204f1b3a5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.000835 8545 tablet_bootstrap.cc:492] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.000926 8545 ts_tablet_manager.cc:1403] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.000s
I20251028 02:59:53.001396 8545 raft_consensus.cc:359] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.001454 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 71feaa313a8f45798561741f7b5d9b20 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.001498 8545 raft_consensus.cc:385] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.001528 8545 raft_consensus.cc:740] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.000424 8546 tablet_bootstrap.cc:492] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.001727 8546 ts_tablet_manager.cc:1403] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:53.002206 8546 raft_consensus.cc:359] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.002293 8546 raft_consensus.cc:385] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.002239 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bb369e38928a4e20af6d144f13efd7ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.002326 8546 raft_consensus.cc:740] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.002442 8546 consensus_queue.cc:260] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:53.002573 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb369e38928a4e20af6d144f13efd7ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.003067 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 1094f4d2872449599e3eaf286def4f18 (DEFAULT_TABLE table=default.Table_f0712855c8944a5bbae3fc9c069b155d [id=c22cf53127924afcbc55ea4a1fd8c04f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.003356 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1094f4d2872449599e3eaf286def4f18. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.003331 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ea6294c26df241f59b9adcc8f2a9eddc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.003667 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea6294c26df241f59b9adcc8f2a9eddc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.004196 8546 ts_tablet_manager.cc:1434] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:53.004315 8656 raft_consensus.cc:493] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.004416 8546 tablet_bootstrap.cc:492] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.004417 8656 raft_consensus.cc:515] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.004802 8656 leader_election.cc:290] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:53.005771 8083 leader_election.cc:343] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 8b88fda3f40540f3b42ee7f33dd33251
W20251028 02:59:53.006057 8083 leader_election.cc:343] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: 8b88fda3f40540f3b42ee7f33dd33251
I20251028 02:59:53.006115 8083 leader_election.cc:304] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:53.006314 8656 raft_consensus.cc:2749] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.006372 8546 tablet_bootstrap.cc:654] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.007863 8401 tablet_service.cc:1505] Processing CreateTablet for tablet e6f9b707ec0345249c3efec1c24a956e (DEFAULT_TABLE table=default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.008140 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6f9b707ec0345249c3efec1c24a956e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.008652 8262 tablet_service.cc:1505] Processing CreateTablet for tablet cdd92cd8a5194e22a91e59aaa4793135 (DEFAULT_TABLE table=default.Table_54bb44e4b3aa4babb80308231391809c [id=2bd44c4bfd2741c7bec6f973ffc7682d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.008903 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd92cd8a5194e22a91e59aaa4793135. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.010006 8399 tablet_service.cc:1505] Processing CreateTablet for tablet cdd92cd8a5194e22a91e59aaa4793135 (DEFAULT_TABLE table=default.Table_54bb44e4b3aa4babb80308231391809c [id=2bd44c4bfd2741c7bec6f973ffc7682d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.010241 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd92cd8a5194e22a91e59aaa4793135. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.011564 8544 tablet_bootstrap.cc:492] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.012629 8544 tablet_bootstrap.cc:654] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.013761 8259 tablet_service.cc:1505] Processing CreateTablet for tablet e6f9b707ec0345249c3efec1c24a956e (DEFAULT_TABLE table=default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.014062 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6f9b707ec0345249c3efec1c24a956e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.014153 8556 raft_consensus.cc:493] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.014235 8556 raft_consensus.cc:515] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.014622 8556 leader_election.cc:290] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.015029 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e359f0121c334cc1a8b3eedbbe64bda2" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:53.015519 8220 leader_election.cc:343] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.015856 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e359f0121c334cc1a8b3eedbbe64bda2" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:53.016229 8218 leader_election.cc:343] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.016294 8218 leader_election.cc:304] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.016466 8556 raft_consensus.cc:2749] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.018347 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 2ab414dbe9144aef8fd72a7f5a95577d (DEFAULT_TABLE table=default.Table_44487777ef26464c9cbbb195199247a0 [id=f04d44307e3c4c07a2403477d11e9fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.018603 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ab414dbe9144aef8fd72a7f5a95577d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.021438 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 1ba114e5a94542a9b3662f28a93cd23a (table default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]) successfully deleted
I20251028 02:59:53.021983 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 2ab414dbe9144aef8fd72a7f5a95577d (DEFAULT_TABLE table=default.Table_44487777ef26464c9cbbb195199247a0 [id=f04d44307e3c4c07a2403477d11e9fcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.022253 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ab414dbe9144aef8fd72a7f5a95577d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.022430 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 71feaa313a8f45798561741f7b5d9b20
I20251028 02:59:53.022691 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 71feaa313a8f45798561741f7b5d9b20 deleted and replaced by tablet 617ca0f545824d14ba7270619f121bcc
I20251028 02:59:53.024205 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 99968de685b44cbfa4c002a6c40e9cb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e9234b72f4448089fe7ff6ba49483b7) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.024394 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 127e08303c564348b14257ec243418f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.024425 8551 tablet_replica.cc:333] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.024595 8551 raft_consensus.cc:2243] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.024546 8545 consensus_queue.cc:260] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.024708 8551 raft_consensus.cc:2272] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.024941 8545 ts_tablet_manager.cc:1434] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.024s user 0.001s sys 0.000s
I20251028 02:59:53.025300 8551 ts_tablet_manager.cc:1916] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.025252 7961 catalog_manager.cc:5649] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:53.025406 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 0703e18d7dba4d628e3cc310be55fcf5 (table default.Table_8fdeffc6f6234305a829715139f150db [id=d4eaebd005bc42dd86d7125ef555546f]). Expected version 2 got 0
I20251028 02:59:53.025522 7961 catalog_manager.cc:5649] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:53.025954 8545 tablet_bootstrap.cc:492] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.026880 8545 tablet_bootstrap.cc:654] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.027701 8551 ts_tablet_manager.cc:1929] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.027839 8551 log.cc:1199] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/99968de685b44cbfa4c002a6c40e9cb3
I20251028 02:59:53.028077 8551 ts_tablet_manager.cc:1950] T 99968de685b44cbfa4c002a6c40e9cb3 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.028188 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 0141ebd1ddad421bbb8eca7615e78351 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.028302 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 127e08303c564348b14257ec243418f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.028435 8552 tablet_replica.cc:333] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.028563 8552 raft_consensus.cc:2243] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.028627 8551 tablet_replica.cc:333] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.028667 8552 raft_consensus.cc:2272] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.029028 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet cf90b539dc8f47b0947d4e2ca859668a from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.029174 8552 ts_tablet_manager.cc:1916] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.029253 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_916c70151f044ff3b34a8e5313675a4f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.029676 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_916c70151f044ff3b34a8e5313675a4f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.029776 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 99968de685b44cbfa4c002a6c40e9cb3 (table default.Table_3d8776d793a54bd795fb38bd45442f51 [id=be1f381c3ca048d891f5d175fa72fc65]) successfully deleted
I20251028 02:59:53.031497 8546 tablet_bootstrap.cc:492] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.031491 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d5e26eda7deb4e4f9d3a93ac052b01c9" } new_table_name: "default.Table_9090533beeb54c4cb4a11365200d3b4e" modify_external_catalogs: true
I20251028 02:59:53.031579 8546 ts_tablet_manager.cc:1403] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.001s
I20251028 02:59:53.031670 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_af395b1589f74f4ebe9dd06515819408" } new_table_name: "default.Table_c7263039d1ed41bcb49853404eabc222" modify_external_catalogs: true
I20251028 02:59:53.032094 8546 raft_consensus.cc:359] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.032191 8546 raft_consensus.cc:385] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.032224 8546 raft_consensus.cc:740] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.032346 8546 consensus_queue.cc:260] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.032527 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 71feaa313a8f45798561741f7b5d9b20 with delete_type TABLET_DATA_DELETED (replaced by tablet 617ca0f545824d14ba7270619f121bcc) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.032491 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2d9658afb66c44639064e40eaf4fa075"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.032768 8546 ts_tablet_manager.cc:1434] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 02:59:53.032831 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d9658afb66c44639064e40eaf4fa075 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.032984 8546 tablet_bootstrap.cc:492] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.034817 8546 tablet_bootstrap.cc:654] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.034876 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 71feaa313a8f45798561741f7b5d9b20 with delete_type TABLET_DATA_DELETED (replaced by tablet 617ca0f545824d14ba7270619f121bcc) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.035043 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 8b88fda3f40540f3b42ee7f33dd33251 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.035220 8552 ts_tablet_manager.cc:1929] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.035280 8552 log.cc:1199] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/127e08303c564348b14257ec243418f4
I20251028 02:59:53.035521 8552 ts_tablet_manager.cc:1950] T 127e08303c564348b14257ec243418f4 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.036018 8552 tablet_replica.cc:333] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.036154 8552 raft_consensus.cc:2243] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.036244 8552 raft_consensus.cc:2272] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.028734 8551 raft_consensus.cc:2243] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.030156 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 71feaa313a8f45798561741f7b5d9b20 with delete_type TABLET_DATA_DELETED (replaced by tablet 617ca0f545824d14ba7270619f121bcc) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.036883 8551 raft_consensus.cc:2272] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.037276 8552 ts_tablet_manager.cc:1916] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.038223 8146 raft_consensus.cc:3060] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.038583 8884 tablet.cc:1722] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.039240 8283 raft_consensus.cc:1275] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:53.039274 8146 raft_consensus.cc:1275] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:53.039628 8551 ts_tablet_manager.cc:1916] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.039980 8648 consensus_queue.cc:1048] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.041461 8616 consensus_queue.cc:1048] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.041707 9023 tablet.cc:1722] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.041940 8753 tablet.cc:1722] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.044587 8551 ts_tablet_manager.cc:1929] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.044652 8551 log.cc:1199] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/127e08303c564348b14257ec243418f4
I20251028 02:59:53.044701 8753 tablet.cc:1722] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:53.044855 8551 ts_tablet_manager.cc:1950] T 127e08303c564348b14257ec243418f4 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.045959 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 127e08303c564348b14257ec243418f4 (table default.Table_24016eb841cb46299d394b16f2d4f3b8 [id=72061aad76324293b5485166aac98a4a]) successfully deleted
I20251028 02:59:53.046118 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 127e08303c564348b14257ec243418f4 (table default.Table_24016eb841cb46299d394b16f2d4f3b8 [id=72061aad76324293b5485166aac98a4a]) successfully deleted
I20251028 02:59:53.048741 8552 ts_tablet_manager.cc:1929] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.048821 8552 log.cc:1199] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/0141ebd1ddad421bbb8eca7615e78351
I20251028 02:59:53.048938 8884 tablet.cc:1722] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:53.049036 8552 ts_tablet_manager.cc:1950] T 0141ebd1ddad421bbb8eca7615e78351 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.052292 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 7801d57550dc40c39f2d3cf7e11e3713 (DEFAULT_TABLE table=default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.052595 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7801d57550dc40c39f2d3cf7e11e3713. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.052861 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 69f1f6895a604b858b28a3e0241245de (DEFAULT_TABLE table=default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.053077 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69f1f6895a604b858b28a3e0241245de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.055038 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 7801d57550dc40c39f2d3cf7e11e3713 (DEFAULT_TABLE table=default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.055366 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7801d57550dc40c39f2d3cf7e11e3713. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.055549 8550 tablet_replica.cc:333] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.055706 8550 raft_consensus.cc:2243] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.055812 8550 pending_rounds.cc:70] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 2 pending ops.
I20251028 02:59:53.055835 8550 pending_rounds.cc:77] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7215597129868120064 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "71feaa313a8f45798561741f7b5d9b20" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_d5e26eda7deb4e4f9d3a93ac052b01c9" dest_uuid: "d7e4cf694f71438298dd32a346abcf75" new_extra_config { } }
I20251028 02:59:53.056010 8550 pending_rounds.cc:77] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597128255868928 op_type: NO_OP noop_request { }
I20251028 02:59:53.056067 8550 raft_consensus.cc:2889] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:53.056118 8550 raft_consensus.cc:2272] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.056716 8552 tablet_replica.cc:333] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.056840 8552 raft_consensus.cc:2243] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.056938 8552 raft_consensus.cc:2272] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.057170 8551 tablet_replica.cc:333] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.057332 8551 raft_consensus.cc:2243] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:53.057535 8551 raft_consensus.cc:2272] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.058681 8551 ts_tablet_manager.cc:1916] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.058765 8261 tablet_service.cc:1505] Processing CreateTablet for tablet de38b2d54cec4f4e892047f4650f611b (DEFAULT_TABLE table=default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.058995 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de38b2d54cec4f4e892047f4650f611b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.059522 8401 tablet_service.cc:1505] Processing CreateTablet for tablet de38b2d54cec4f4e892047f4650f611b (DEFAULT_TABLE table=default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.059770 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de38b2d54cec4f4e892047f4650f611b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.060741 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 69f1f6895a604b858b28a3e0241245de (DEFAULT_TABLE table=default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.060854 8551 ts_tablet_manager.cc:1929] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:53.060920 8551 log.cc:1199] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/71feaa313a8f45798561741f7b5d9b20
I20251028 02:59:53.061014 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69f1f6895a604b858b28a3e0241245de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.061187 8551 ts_tablet_manager.cc:1950] T 71feaa313a8f45798561741f7b5d9b20 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.062081 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 71feaa313a8f45798561741f7b5d9b20 (table default.Table_d5e26eda7deb4e4f9d3a93ac052b01c9 [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
I20251028 02:59:53.063073 8119 tablet_service.cc:1505] Processing CreateTablet for tablet de38b2d54cec4f4e892047f4650f611b (DEFAULT_TABLE table=default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.063386 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de38b2d54cec4f4e892047f4650f611b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.063898 8546 tablet_bootstrap.cc:492] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.063982 8546 ts_tablet_manager.cc:1403] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20251028 02:59:53.064518 8546 raft_consensus.cc:359] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.064631 8546 raft_consensus.cc:385] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.064662 8546 raft_consensus.cc:740] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.064664 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 0141ebd1ddad421bbb8eca7615e78351 (table default.Table_83adc71d84344ff686fc777f2f72d54b [id=0b22f43d0bda44b98e6f4b2f26153c5c]) successfully deleted
I20251028 02:59:53.065094 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet e359f0121c334cc1a8b3eedbbe64bda2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.065148 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 2be7be7c732e4780859cbe6624edf54e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.065207 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.065299 8546 consensus_queue.cc:260] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.065842 8546 ts_tablet_manager.cc:1434] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:53.066032 8546 tablet_bootstrap.cc:492] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.066656 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 69f1f6895a604b858b28a3e0241245de (DEFAULT_TABLE table=default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.066762 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 7801d57550dc40c39f2d3cf7e11e3713 (DEFAULT_TABLE table=default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.066890 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69f1f6895a604b858b28a3e0241245de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.066998 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7801d57550dc40c39f2d3cf7e11e3713. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.067174 8546 tablet_bootstrap.cc:654] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.067380 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 99968de685b44cbfa4c002a6c40e9cb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e9234b72f4448089fe7ff6ba49483b7) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.068286 8546 tablet_bootstrap.cc:492] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.068361 8546 ts_tablet_manager.cc:1403] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:53.068802 8546 raft_consensus.cc:359] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.068897 8546 raft_consensus.cc:385] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.068928 8546 raft_consensus.cc:740] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.069092 8546 consensus_queue.cc:260] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.069391 8546 ts_tablet_manager.cc:1434] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.069636 8546 tablet_bootstrap.cc:492] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.071166 8546 tablet_bootstrap.cc:654] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.071861 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3d8776d793a54bd795fb38bd45442f51" } modify_external_catalogs: true
I20251028 02:59:53.071990 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3d8776d793a54bd795fb38bd45442f51" } modify_external_catalogs: true
I20251028 02:59:53.072849 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0a1e2cd2aace496790ca53a8bdfbea25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.073251 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a1e2cd2aace496790ca53a8bdfbea25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.074920 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4b9a111a46cd448cac4abf954888e213"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.074987 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f0712855c8944a5bbae3fc9c069b155d" } modify_external_catalogs: true
I20251028 02:59:53.075100 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f0712855c8944a5bbae3fc9c069b155d" } modify_external_catalogs: true
W20251028 02:59:53.075253 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b9a111a46cd448cac4abf954888e213 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.077096 8552 ts_tablet_manager.cc:1916] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.079326 8552 ts_tablet_manager.cc:1929] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:53.079401 8552 log.cc:1199] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/71feaa313a8f45798561741f7b5d9b20
I20251028 02:59:53.079605 8552 ts_tablet_manager.cc:1950] T 71feaa313a8f45798561741f7b5d9b20 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.080085 8552 tablet_replica.cc:333] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.080193 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 71feaa313a8f45798561741f7b5d9b20 (table default.Table_d5e26eda7deb4e4f9d3a93ac052b01c9 [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
I20251028 02:59:53.080222 8552 raft_consensus.cc:2243] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.080314 8552 raft_consensus.cc:2272] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.080705 8552 ts_tablet_manager.cc:1916] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:53.081029 9046 consensus_queue.cc:1175] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20251028 02:59:53.082263 8550 ts_tablet_manager.cc:1916] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:53.082844 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 617ca0f545824d14ba7270619f121bcc has no consensus state
I20251028 02:59:53.084048 8148 raft_consensus.cc:3060] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.084646 8550 ts_tablet_manager.cc:1929] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:53.084722 8550 log.cc:1199] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/71feaa313a8f45798561741f7b5d9b20
I20251028 02:59:53.084988 8550 ts_tablet_manager.cc:1950] T 71feaa313a8f45798561741f7b5d9b20 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.085690 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 71feaa313a8f45798561741f7b5d9b20 (table default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
I20251028 02:59:53.086025 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1e9234b72f4448089fe7ff6ba49483b7
I20251028 02:59:53.086092 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet cf90b539dc8f47b0947d4e2ca859668a
W20251028 02:59:53.086239 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c048520fe7b248e586e44a2047ffb239 consensus state has no leader
I20251028 02:59:53.086335 7963 catalog_manager.cc:6719] ReplaceTablet: tablet cf90b539dc8f47b0947d4e2ca859668a deleted and replaced by tablet 4246c4bd2abf4569987ca346996445d4
I20251028 02:59:53.086557 8938 consensus_queue.cc:1048] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.087013 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet cf90b539dc8f47b0947d4e2ca859668a with delete_type TABLET_DATA_DELETED (replaced by tablet 4246c4bd2abf4569987ca346996445d4) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.087148 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e9234b72f4448089fe7ff6ba49483b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.087216 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e9234b72f4448089fe7ff6ba49483b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.087682 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet cf90b539dc8f47b0947d4e2ca859668a with delete_type TABLET_DATA_DELETED (replaced by tablet 4246c4bd2abf4569987ca346996445d4) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:53.087917 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 1e9234b72f4448089fe7ff6ba49483b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e9234b72f4448089fe7ff6ba49483b7 already in progress: creating tablet
I20251028 02:59:53.087987 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet cf90b539dc8f47b0947d4e2ca859668a with delete_type TABLET_DATA_DELETED (replaced by tablet 4246c4bd2abf4569987ca346996445d4) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.088132 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e9234b72f4448089fe7ff6ba49483b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:53.088219 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet cf90b539dc8f47b0947d4e2ca859668a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf90b539dc8f47b0947d4e2ca859668a already in progress: creating tablet
I20251028 02:59:53.088361 8550 tablet_replica.cc:333] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.088505 8550 raft_consensus.cc:2243] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.088608 8550 raft_consensus.cc:2272] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.089565 8552 ts_tablet_manager.cc:1929] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.089639 8552 log.cc:1199] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/e359f0121c334cc1a8b3eedbbe64bda2
I20251028 02:59:53.089867 8552 ts_tablet_manager.cc:1950] T e359f0121c334cc1a8b3eedbbe64bda2 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.090282 8550 ts_tablet_manager.cc:1916] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.092434 7965 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52] alter complete (version 3)
I20251028 02:59:53.093590 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f38ce803e114831ace1412abdad7126 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:51 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.093719 8616 consensus_queue.cc:1048] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.093932 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 71feaa313a8f45798561741f7b5d9b20 with delete_type TABLET_DATA_DELETED (replaced by tablet 617ca0f545824d14ba7270619f121bcc) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.094348 8552 tablet_replica.cc:333] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.094486 8552 raft_consensus.cc:2243] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.094579 8552 raft_consensus.cc:2272] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.095791 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet e359f0121c334cc1a8b3eedbbe64bda2 (table default.Table_726e0c2d9d4044d8b79bb266e21a8d9c [id=6630b83265864f36ad74cac692f131e3]) successfully deleted
I20251028 02:59:53.095947 8550 ts_tablet_manager.cc:1929] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.096024 8550 log.cc:1199] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8b88fda3f40540f3b42ee7f33dd33251
I20251028 02:59:53.096275 8550 ts_tablet_manager.cc:1950] T 8b88fda3f40540f3b42ee7f33dd33251 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.097291 8552 ts_tablet_manager.cc:1916] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.097788 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 71feaa313a8f45798561741f7b5d9b20 with delete_type TABLET_DATA_DELETED (replaced by tablet 617ca0f545824d14ba7270619f121bcc) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:53.097990 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 71feaa313a8f45798561741f7b5d9b20 because the tablet was not found. No further retry: Not found: Tablet not found: 71feaa313a8f45798561741f7b5d9b20
I20251028 02:59:53.098349 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 9c1d8552cc1941bda2e9f02fba56c9f9 (table default.Table_bb551b322c5140ba89d154ff450085dd [id=4e8c5072da4e4ec6a616ba4c7fc91cd3]). Expected version 1 got 0
I20251028 02:59:53.099792 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d711941006d4417901135b7761a251f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.099895 8552 ts_tablet_manager.cc:1929] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.099960 8552 log.cc:1199] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/2be7be7c732e4780859cbe6624edf54e
I20251028 02:59:53.100185 8552 ts_tablet_manager.cc:1950] T 2be7be7c732e4780859cbe6624edf54e P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.100440 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8b88fda3f40540f3b42ee7f33dd33251 (table default.Table_7fb744c8c91e4c9daadacdab1a408fa1 [id=98a07edd5f664674a2d33344038a8801]) successfully deleted
I20251028 02:59:53.100694 8552 tablet_replica.cc:333] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.100843 8552 raft_consensus.cc:2243] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.100952 8552 raft_consensus.cc:2272] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.101177 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 2be7be7c732e4780859cbe6624edf54e (table default.Table_2a09149ed84d4343a81458e2fb78f262 [id=2bcc83cc472b4ec5a7ca79a8a7c5ff23]) successfully deleted
I20251028 02:59:53.101744 8552 ts_tablet_manager.cc:1916] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:53.101938 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet cf90b539dc8f47b0947d4e2ca859668a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf90b539dc8f47b0947d4e2ca859668a already in progress: creating tablet
I20251028 02:59:53.102702 8544 tablet_bootstrap.cc:492] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.102702 8545 tablet_bootstrap.cc:492] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.102810 8544 ts_tablet_manager.cc:1403] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.091s user 0.000s sys 0.002s
I20251028 02:59:53.102802 8545 ts_tablet_manager.cc:1403] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.077s user 0.001s sys 0.000s
I20251028 02:59:53.103271 8545 raft_consensus.cc:359] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.103271 8544 raft_consensus.cc:359] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.103436 8545 raft_consensus.cc:385] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.103444 8544 raft_consensus.cc:385] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.103482 8544 raft_consensus.cc:740] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.103482 8545 raft_consensus.cc:740] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.103610 8545 consensus_queue.cc:260] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.103610 8544 consensus_queue.cc:260] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.103914 8545 ts_tablet_manager.cc:1434] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.103981 8544 ts_tablet_manager.cc:1434] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.104146 8544 tablet_bootstrap.cc:492] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.104146 8545 tablet_bootstrap.cc:492] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:53.104655 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1e9234b72f4448089fe7ff6ba49483b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e9234b72f4448089fe7ff6ba49483b7 already in progress: creating tablet
I20251028 02:59:53.105086 8550 tablet_replica.cc:333] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.105098 8545 tablet_bootstrap.cc:654] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.105235 8550 raft_consensus.cc:2243] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.105350 8550 raft_consensus.cc:2272] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.105366 8552 ts_tablet_manager.cc:1929] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.105516 8552 log.cc:1199] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/2bcbf3c72bb34c6291f4c1b1b9b1e906
I20251028 02:59:53.105774 8552 ts_tablet_manager.cc:1950] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.106202 8544 tablet_bootstrap.cc:654] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.106619 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 (table default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]) successfully deleted
I20251028 02:59:53.106890 8552 tablet_replica.cc:333] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.107059 8552 raft_consensus.cc:2243] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.107227 8552 raft_consensus.cc:2272] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.107676 8552 ts_tablet_manager.cc:1916] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.109972 8552 ts_tablet_manager.cc:1929] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.110112 8552 log.cc:1199] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/99968de685b44cbfa4c002a6c40e9cb3
I20251028 02:59:53.110388 8552 ts_tablet_manager.cc:1950] T 99968de685b44cbfa4c002a6c40e9cb3 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.110541 8965 raft_consensus.cc:493] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.110637 8965 raft_consensus.cc:515] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.111074 8965 leader_election.cc:290] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.111254 8552 tablet_replica.cc:333] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.111424 8552 raft_consensus.cc:2243] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:53.111449 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet cf90b539dc8f47b0947d4e2ca859668a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf90b539dc8f47b0947d4e2ca859668a already in progress: creating tablet
I20251028 02:59:53.111523 8552 raft_consensus.cc:2272] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:53.111598 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 1e9234b72f4448089fe7ff6ba49483b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e9234b72f4448089fe7ff6ba49483b7 already in progress: creating tablet
I20251028 02:59:53.111632 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3057f4749edd43ac889f64d1f4e282d1" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.111688 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3057f4749edd43ac889f64d1f4e282d1" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.111758 8420 raft_consensus.cc:2468] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
W20251028 02:59:53.111829 7955 catalog_manager.cc:4952] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 71feaa313a8f45798561741f7b5d9b20 because the tablet was not found. No further retry: Not found: Tablet not found: 71feaa313a8f45798561741f7b5d9b20
I20251028 02:59:53.112213 8220 leader_election.cc:304] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.112262 8552 ts_tablet_manager.cc:1916] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:53.112449 8218 leader_election.cc:343] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.112666 8965 raft_consensus.cc:2804] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.112718 8965 raft_consensus.cc:493] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.112756 8965 raft_consensus.cc:3060] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.113742 8965 raft_consensus.cc:515] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.114117 8965 leader_election.cc:290] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.114280 8552 ts_tablet_manager.cc:1929] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.114339 8552 log.cc:1199] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/2d711941006d4417901135b7761a251f
I20251028 02:59:53.114548 8552 ts_tablet_manager.cc:1950] T 2d711941006d4417901135b7761a251f P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.114881 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3057f4749edd43ac889f64d1f4e282d1" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:53.115010 8420 raft_consensus.cc:3060] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.115706 8550 ts_tablet_manager.cc:1916] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.115898 8420 raft_consensus.cc:2468] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:53.116310 8220 leader_election.cc:304] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.116513 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1094f4d2872449599e3eaf286def4f18
I20251028 02:59:53.116518 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3057f4749edd43ac889f64d1f4e282d1" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:53.116753 8556 consensus_queue.cc:1048] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.117121 8556 raft_consensus.cc:2804] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:53.117214 8556 raft_consensus.cc:697] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:53.117373 8556 consensus_queue.cc:237] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.118319 8545 tablet_bootstrap.cc:492] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.118409 8545 ts_tablet_manager.cc:1403] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20251028 02:59:53.118873 8545 raft_consensus.cc:359] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.118958 8545 raft_consensus.cc:385] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.118979 8545 raft_consensus.cc:740] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.119097 8545 consensus_queue.cc:260] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.120074 8545 ts_tablet_manager.cc:1434] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:53.120326 8545 tablet_bootstrap.cc:492] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.120990 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 1094f4d2872449599e3eaf286def4f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.121410 8545 tablet_bootstrap.cc:654] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:53.121536 8218 leader_election.cc:343] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:53.121899 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 1094f4d2872449599e3eaf286def4f18 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1094f4d2872449599e3eaf286def4f18 already in progress: creating tablet
I20251028 02:59:53.121913 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 1094f4d2872449599e3eaf286def4f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.122094 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 2d711941006d4417901135b7761a251f (table default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]) successfully deleted
I20251028 02:59:53.122272 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 1094f4d2872449599e3eaf286def4f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.124089 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 1ba114e5a94542a9b3662f28a93cd23a with delete_type TABLET_DATA_DELETED (replaced by tablet 69f1f6895a604b858b28a3e0241245de) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.125123 8545 tablet_bootstrap.cc:492] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.125217 8545 ts_tablet_manager.cc:1403] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:53.125634 8545 raft_consensus.cc:359] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.125746 8545 raft_consensus.cc:385] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251028 02:59:53.125736 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 1094f4d2872449599e3eaf286def4f18 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1094f4d2872449599e3eaf286def4f18 already in progress: creating tablet
I20251028 02:59:53.125788 8545 raft_consensus.cc:740] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.125918 8545 consensus_queue.cc:260] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.126101 8550 ts_tablet_manager.cc:1929] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.126186 8550 log.cc:1199] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/6f38ce803e114831ace1412abdad7126
I20251028 02:59:53.126223 8545 ts_tablet_manager.cc:1434] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.126431 8545 tablet_bootstrap.cc:492] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.126439 8550 ts_tablet_manager.cc:1950] T 6f38ce803e114831ace1412abdad7126 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.127175 8550 tablet_replica.cc:333] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.127327 8550 raft_consensus.cc:2243] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.127424 8550 raft_consensus.cc:2272] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.128212 8546 tablet_bootstrap.cc:492] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.128304 8546 ts_tablet_manager.cc:1403] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.059s user 0.000s sys 0.002s
I20251028 02:59:53.128722 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 6f38ce803e114831ace1412abdad7126 (table default.Table_828e8df9af6a48168a189935f63b2941 [id=42c89dfccc714a9eb42e3779130c8af4]) successfully deleted
I20251028 02:59:53.128765 8546 raft_consensus.cc:359] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.128887 8546 raft_consensus.cc:385] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251028 02:59:53.128885 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1094f4d2872449599e3eaf286def4f18 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1094f4d2872449599e3eaf286def4f18 already in progress: creating tablet
I20251028 02:59:53.128924 8546 raft_consensus.cc:740] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.129051 8546 consensus_queue.cc:260] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.129316 8546 ts_tablet_manager.cc:1434] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:53.129824 8545 tablet_bootstrap.cc:654] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.129844 8546 tablet_bootstrap.cc:492] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.130887 8546 tablet_bootstrap.cc:654] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.131601 8550 ts_tablet_manager.cc:1916] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.131990 8544 tablet_bootstrap.cc:492] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.132079 8544 ts_tablet_manager.cc:1403] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20251028 02:59:53.132274 8545 tablet_bootstrap.cc:492] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.132351 8545 ts_tablet_manager.cc:1403] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:53.132537 8544 raft_consensus.cc:359] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.132638 8544 raft_consensus.cc:385] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.132669 8544 raft_consensus.cc:740] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.132789 8544 consensus_queue.cc:260] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.133108 8544 ts_tablet_manager.cc:1434] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.133320 8544 tablet_bootstrap.cc:492] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.134380 8544 tablet_bootstrap.cc:654] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.134392 8556 consensus_queue.cc:1048] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.132815 8545 raft_consensus.cc:359] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.135465 8545 raft_consensus.cc:385] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.135512 8545 raft_consensus.cc:740] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.136974 8544 tablet_bootstrap.cc:492] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.137063 8544 ts_tablet_manager.cc:1403] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:53.137530 8544 raft_consensus.cc:359] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.137634 8544 raft_consensus.cc:385] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.137665 8544 raft_consensus.cc:740] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.137673 8550 ts_tablet_manager.cc:1929] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.137746 8550 log.cc:1199] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/1ba114e5a94542a9b3662f28a93cd23a
I20251028 02:59:53.137810 8544 consensus_queue.cc:260] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.137995 8550 ts_tablet_manager.cc:1950] T 1ba114e5a94542a9b3662f28a93cd23a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.138110 8544 ts_tablet_manager.cc:1434] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.138986 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e69b7cea080549b1b8a67cb91703a472" } modify_external_catalogs: true
I20251028 02:59:53.139070 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b55d0b45f7e543e8a5927bde50d0502b" } new_table_name: "default.Table_5da94c10f01846a59e4ce4ff530330d3" modify_external_catalogs: true
I20251028 02:59:53.139109 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e69b7cea080549b1b8a67cb91703a472" } modify_external_catalogs: true
I20251028 02:59:53.139981 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_afc8d935c16a4f0b9d3bff7efa913adf" } modify_external_catalogs: true
I20251028 02:59:53.140080 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_afc8d935c16a4f0b9d3bff7efa913adf" } modify_external_catalogs: true
I20251028 02:59:53.140915 8545 consensus_queue.cc:260] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.141269 8545 ts_tablet_manager.cc:1434] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20251028 02:59:53.141485 8545 tablet_bootstrap.cc:492] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.143572 8553 raft_consensus.cc:493] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.143683 8553 raft_consensus.cc:515] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.144093 8553 leader_election.cc:290] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.144534 8912 raft_consensus.cc:493] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.144603 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9b92443ae55f4a489d92d6175b03f377" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.144627 8912 raft_consensus.cc:515] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.144730 8420 raft_consensus.cc:2468] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:53.144994 8912 leader_election.cc:290] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.145045 8220 leader_election.cc:304] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.145236 8556 raft_consensus.cc:2804] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.145298 8556 raft_consensus.cc:493] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.145352 8556 raft_consensus.cc:3060] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.145660 8545 tablet_bootstrap.cc:654] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.146409 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f1c1e14665c4712bffa439ab98b2a6e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.146437 8556 raft_consensus.cc:515] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.146556 8420 raft_consensus.cc:2393] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:53.146788 8556 leader_election.cc:290] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.146833 8544 tablet_bootstrap.cc:492] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.146948 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet d248313db35e40e38e2c5e07e68a69af from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.147096 8545 tablet_bootstrap.cc:492] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.147167 8545 ts_tablet_manager.cc:1403] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20251028 02:59:53.147601 8545 raft_consensus.cc:359] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.147696 8545 raft_consensus.cc:385] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.147720 8545 raft_consensus.cc:740] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.147810 8545 consensus_queue.cc:260] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.147984 8544 tablet_bootstrap.cc:654] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.148084 8545 ts_tablet_manager.cc:1434] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.148237 8545 tablet_bootstrap.cc:492] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.148370 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9b92443ae55f4a489d92d6175b03f377" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:53.148463 8420 raft_consensus.cc:3060] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.148615 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f1c1e14665c4712bffa439ab98b2a6e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.149235 8545 tablet_bootstrap.cc:654] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.149434 8083 leader_election.cc:304] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:53.149536 8420 raft_consensus.cc:2468] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:53.150038 8912 raft_consensus.cc:3060] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.150312 8545 tablet_bootstrap.cc:492] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.150380 8545 ts_tablet_manager.cc:1403] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:53.150827 8545 raft_consensus.cc:359] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.150918 8545 raft_consensus.cc:385] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.150960 8545 raft_consensus.cc:740] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.151084 8912 raft_consensus.cc:2749] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.151110 8545 consensus_queue.cc:260] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.151400 8545 ts_tablet_manager.cc:1434] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.151558 8545 tablet_bootstrap.cc:492] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.151539 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 1ba114e5a94542a9b3662f28a93cd23a (table default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]) successfully deleted
I20251028 02:59:53.151871 8220 leader_election.cc:304] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.152216 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9b92443ae55f4a489d92d6175b03f377" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.152365 8556 raft_consensus.cc:2804] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:53.152455 8556 raft_consensus.cc:697] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:53.152590 8556 consensus_queue.cc:237] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.152719 8545 tablet_bootstrap.cc:654] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.152712 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9b92443ae55f4a489d92d6175b03f377" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:53.153585 8544 tablet_bootstrap.cc:492] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.153676 8544 ts_tablet_manager.cc:1403] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
W20251028 02:59:53.153766 8218 leader_election.cc:343] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:53.153955 8218 leader_election.cc:343] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.154183 8544 raft_consensus.cc:359] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.154276 8544 raft_consensus.cc:385] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.154309 8544 raft_consensus.cc:740] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.154404 8544 consensus_queue.cc:260] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.154685 8544 ts_tablet_manager.cc:1434] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:53.154898 8544 tablet_bootstrap.cc:492] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.156728 8544 tablet_bootstrap.cc:654] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.157393 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 3410b1b517b146d4b38cb47862c85e68 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.157958 8544 tablet_bootstrap.cc:492] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.158084 8544 ts_tablet_manager.cc:1403] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20251028 02:59:53.158311 8399 tablet_service.cc:1505] Processing CreateTablet for tablet d81b3fe6073d4380882057eb97907fbb (DEFAULT_TABLE table=default.Table_916c70151f044ff3b34a8e5313675a4f [id=ed2bacfbb3f24d5490ad58320bb33656]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.158583 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d81b3fe6073d4380882057eb97907fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.158610 8544 raft_consensus.cc:359] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.158706 8544 raft_consensus.cc:385] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.158736 8544 raft_consensus.cc:740] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.158882 8544 consensus_queue.cc:260] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.159519 7961 catalog_manager.cc:5649] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:53.160373 8545 tablet_bootstrap.cc:492] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.160454 8545 ts_tablet_manager.cc:1403] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 02:59:53.160446 8544 ts_tablet_manager.cc:1434] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:53.160629 8544 tablet_bootstrap.cc:492] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.160921 8545 raft_consensus.cc:359] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.161024 8545 raft_consensus.cc:385] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.161070 8545 raft_consensus.cc:740] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.161197 8545 consensus_queue.cc:260] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.161482 8545 ts_tablet_manager.cc:1434] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.161645 8545 tablet_bootstrap.cc:492] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.161743 8544 tablet_bootstrap.cc:654] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.161969 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 5fd8682b26c544d58c79f22dd469eedc (DEFAULT_TABLE table=default.Table_f3ea294303c54da8845388204f1b3a5b [id=0cd346d2b86d4c2494a2289ae312dca7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.162200 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd8682b26c544d58c79f22dd469eedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.162586 8545 tablet_bootstrap.cc:654] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.164033 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b (DEFAULT_TABLE table=default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.164253 8545 tablet_bootstrap.cc:492] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.164276 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.164327 8545 ts_tablet_manager.cc:1403] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:53.164778 8545 raft_consensus.cc:359] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.165174 8261 tablet_service.cc:1505] Processing CreateTablet for tablet d81b3fe6073d4380882057eb97907fbb (DEFAULT_TABLE table=default.Table_916c70151f044ff3b34a8e5313675a4f [id=ed2bacfbb3f24d5490ad58320bb33656]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.165254 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 5fd8682b26c544d58c79f22dd469eedc (DEFAULT_TABLE table=default.Table_f3ea294303c54da8845388204f1b3a5b [id=0cd346d2b86d4c2494a2289ae312dca7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.165426 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d81b3fe6073d4380882057eb97907fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.165558 8545 raft_consensus.cc:385] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.165612 8545 raft_consensus.cc:740] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.165635 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd8682b26c544d58c79f22dd469eedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.165724 8545 consensus_queue.cc:260] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.166105 8545 ts_tablet_manager.cc:1434] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:53.166260 8545 tablet_bootstrap.cc:492] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.166404 8262 tablet_service.cc:1505] Processing CreateTablet for tablet ec69044a4c3e4f049f90be835cf73367 (DEFAULT_TABLE table=default.Table_cc38c53f9a064f3f8af6b4d688c1aada [id=dbed4c8748394b8eb5e82d478f195bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.166628 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec69044a4c3e4f049f90be835cf73367. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.167259 8545 tablet_bootstrap.cc:654] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.169384 8545 tablet_bootstrap.cc:492] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.169479 8545 ts_tablet_manager.cc:1403] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:53.169989 8545 raft_consensus.cc:359] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.170099 8545 raft_consensus.cc:385] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.170132 8545 raft_consensus.cc:740] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.170548 8545 consensus_queue.cc:260] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.171739 8545 ts_tablet_manager.cc:1434] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:53.171747 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet e6f9b707ec0345249c3efec1c24a956e from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.171933 8545 tablet_bootstrap.cc:492] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.172225 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 12dce4bb3d4b43608d7da9350b14e1d0 (DEFAULT_TABLE table=default.Table_f2b0692abe27469a8f0728978e62ae26 [id=5dd4cb94fa384facbdd1aef677a4b498]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.172497 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12dce4bb3d4b43608d7da9350b14e1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.172901 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b (DEFAULT_TABLE table=default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.173111 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.173821 8545 tablet_bootstrap.cc:654] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.173955 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 12dce4bb3d4b43608d7da9350b14e1d0 (DEFAULT_TABLE table=default.Table_f2b0692abe27469a8f0728978e62ae26 [id=5dd4cb94fa384facbdd1aef677a4b498]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.174193 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12dce4bb3d4b43608d7da9350b14e1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.174644 8123 tablet_service.cc:1505] Processing CreateTablet for tablet d81b3fe6073d4380882057eb97907fbb (DEFAULT_TABLE table=default.Table_916c70151f044ff3b34a8e5313675a4f [id=ed2bacfbb3f24d5490ad58320bb33656]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.174831 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 617ca0f545824d14ba7270619f121bcc (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.174912 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 12dce4bb3d4b43608d7da9350b14e1d0 (DEFAULT_TABLE table=default.Table_f2b0692abe27469a8f0728978e62ae26 [id=5dd4cb94fa384facbdd1aef677a4b498]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.175060 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 5fd8682b26c544d58c79f22dd469eedc (DEFAULT_TABLE table=default.Table_f3ea294303c54da8845388204f1b3a5b [id=0cd346d2b86d4c2494a2289ae312dca7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.175045 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 617ca0f545824d14ba7270619f121bcc (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.175269 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617ca0f545824d14ba7270619f121bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.175459 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b (DEFAULT_TABLE table=default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.175675 8544 tablet_bootstrap.cc:492] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.175745 8117 tablet_service.cc:1505] Processing CreateTablet for tablet ec69044a4c3e4f049f90be835cf73367 (DEFAULT_TABLE table=default.Table_cc38c53f9a064f3f8af6b4d688c1aada [id=dbed4c8748394b8eb5e82d478f195bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.175796 8544 ts_tablet_manager.cc:1403] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 02:59:53.174881 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d81b3fe6073d4380882057eb97907fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.176112 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617ca0f545824d14ba7270619f121bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.176326 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.176587 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12dce4bb3d4b43608d7da9350b14e1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.177651 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 617ca0f545824d14ba7270619f121bcc (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.177896 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617ca0f545824d14ba7270619f121bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.178135 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd8682b26c544d58c79f22dd469eedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.178469 8544 raft_consensus.cc:359] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.178561 8544 raft_consensus.cc:385] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.178598 8544 raft_consensus.cc:740] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.178706 8544 consensus_queue.cc:260] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.178996 8544 ts_tablet_manager.cc:1434] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20251028 02:59:53.179200 8544 tablet_bootstrap.cc:492] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.179452 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec69044a4c3e4f049f90be835cf73367. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.169384 8399 tablet_service.cc:1505] Processing CreateTablet for tablet ec69044a4c3e4f049f90be835cf73367 (DEFAULT_TABLE table=default.Table_cc38c53f9a064f3f8af6b4d688c1aada [id=dbed4c8748394b8eb5e82d478f195bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.179811 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec69044a4c3e4f049f90be835cf73367. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.180330 8544 tablet_bootstrap.cc:654] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.181457 8545 tablet_bootstrap.cc:492] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.181545 8545 ts_tablet_manager.cc:1403] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:53.182024 8545 raft_consensus.cc:359] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.182134 8545 raft_consensus.cc:385] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.182166 8545 raft_consensus.cc:740] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.182288 8545 consensus_queue.cc:260] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.182575 8545 ts_tablet_manager.cc:1434] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.183090 8616 raft_consensus.cc:493] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.183159 8616 raft_consensus.cc:515] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.183553 8616 leader_election.cc:290] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.183769 8545 tablet_bootstrap.cc:492] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.183921 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99c37c774ae2458989377a8bac94f4e8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.184113 8283 raft_consensus.cc:2468] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:53.184414 8356 leader_election.cc:304] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.184580 8648 raft_consensus.cc:2804] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.184631 8648 raft_consensus.cc:493] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.184670 8648 raft_consensus.cc:3060] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.184615 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d2e5ac67a4414c03b7890dea514caeee" } modify_external_catalogs: true
I20251028 02:59:53.184747 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d2e5ac67a4414c03b7890dea514caeee" } modify_external_catalogs: true
I20251028 02:59:53.184893 8545 tablet_bootstrap.cc:654] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.185369 8616 raft_consensus.cc:493] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.185446 8616 raft_consensus.cc:515] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.185590 8648 raft_consensus.cc:515] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.185791 8616 leader_election.cc:290] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.185830 8648 leader_election.cc:290] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.186403 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f57b4fa3ceec4557bc6d6a2125d80790" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.186525 8283 raft_consensus.cc:2468] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:53.186522 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99c37c774ae2458989377a8bac94f4e8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:53.186671 8285 raft_consensus.cc:3060] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.186932 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99c37c774ae2458989377a8bac94f4e8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.187040 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f57b4fa3ceec4557bc6d6a2125d80790" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.187189 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99c37c774ae2458989377a8bac94f4e8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:53.187495 8356 leader_election.cc:304] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.187712 8285 raft_consensus.cc:2468] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:53.187875 8616 raft_consensus.cc:2804] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.187932 8616 raft_consensus.cc:493] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.187978 8616 raft_consensus.cc:3060] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:53.188174 8354 leader_election.cc:343] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:53.188344 8354 leader_election.cc:343] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:53.188478 8354 leader_election.cc:343] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.188854 8616 raft_consensus.cc:515] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.189188 8616 leader_election.cc:290] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.189688 8356 leader_election.cc:304] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.189929 8648 raft_consensus.cc:2804] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:53.190004 8648 raft_consensus.cc:697] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:53.190136 8648 consensus_queue.cc:237] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:53.192301 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 03959a61996b40d8b9ef8b4b9708f017 consensus state has no leader
I20251028 02:59:53.192713 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f57b4fa3ceec4557bc6d6a2125d80790" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:53.192847 8283 raft_consensus.cc:3060] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.193874 8283 raft_consensus.cc:2468] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:53.194447 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f57b4fa3ceec4557bc6d6a2125d80790" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:53.194802 8549 raft_consensus.cc:493] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251028 02:59:53.194809 8354 leader_election.cc:343] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.194905 8549 raft_consensus.cc:515] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.194924 8356 leader_election.cc:304] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.195230 8938 raft_consensus.cc:2804] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:53.195358 8938 raft_consensus.cc:697] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:53.195474 8938 consensus_queue.cc:237] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.195636 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e359f0121c334cc1a8b3eedbbe64bda2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.195238 8549 leader_election.cc:290] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 02:59:53.196421 8354 leader_election.cc:343] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.196918 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d248313db35e40e38e2c5e07e68a69af
I20251028 02:59:53.197197 7967 catalog_manager.cc:6719] ReplaceTablet: tablet d248313db35e40e38e2c5e07e68a69af deleted and replaced by tablet d9c54e91d2d94573b40b50752318536d
I20251028 02:59:53.197984 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet d248313db35e40e38e2c5e07e68a69af with delete_type TABLET_DATA_DELETED (replaced by tablet d9c54e91d2d94573b40b50752318536d) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.198098 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 429b9cd6b775436a865afd64ec5eae4f (DEFAULT_TABLE table=default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.198158 8551 tablet_replica.cc:333] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.198328 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet d248313db35e40e38e2c5e07e68a69af with delete_type TABLET_DATA_DELETED (replaced by tablet d9c54e91d2d94573b40b50752318536d) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.198462 8551 raft_consensus.cc:2243] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:53.198467 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429b9cd6b775436a865afd64ec5eae4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.198689 8551 raft_consensus.cc:2272] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.198902 8552 tablet_replica.cc:333] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.199048 8552 raft_consensus.cc:2243] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.199168 8552 raft_consensus.cc:2272] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.199918 8551 ts_tablet_manager.cc:1916] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.200518 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 429b9cd6b775436a865afd64ec5eae4f (DEFAULT_TABLE table=default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.200769 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429b9cd6b775436a865afd64ec5eae4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.201833 8262 tablet_service.cc:1505] Processing CreateTablet for tablet ebc0e6d526364d4ba37e443d1e681fbc (DEFAULT_TABLE table=default.Table_4f4721dca71f432b82e320f8fa9b0cf5 [id=bc3f005bc9424552831e2ecca5d049a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.202090 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc0e6d526364d4ba37e443d1e681fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.202106 8551 ts_tablet_manager.cc:1929] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.202162 8551 log.cc:1199] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/d248313db35e40e38e2c5e07e68a69af
I20251028 02:59:53.202414 8551 ts_tablet_manager.cc:1950] T d248313db35e40e38e2c5e07e68a69af P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 02:59:53.203702 8356 leader_election.cc:343] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: e359f0121c334cc1a8b3eedbbe64bda2
I20251028 02:59:53.203781 8356 leader_election.cc:304] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.204075 8394 tablet_service.cc:1505] Processing CreateTablet for tablet ebc0e6d526364d4ba37e443d1e681fbc (DEFAULT_TABLE table=default.Table_4f4721dca71f432b82e320f8fa9b0cf5 [id=bc3f005bc9424552831e2ecca5d049a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.204370 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc0e6d526364d4ba37e443d1e681fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.204464 8616 raft_consensus.cc:2749] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.205466 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 02f6a32db68e4c609da39c3ee6c97348
I20251028 02:59:53.206496 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 429b9cd6b775436a865afd64ec5eae4f (DEFAULT_TABLE table=default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.206629 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 02f6a32db68e4c609da39c3ee6c97348 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.206761 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429b9cd6b775436a865afd64ec5eae4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.207163 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet d248313db35e40e38e2c5e07e68a69af (table default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]) successfully deleted
I20251028 02:59:53.207554 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 02f6a32db68e4c609da39c3ee6c97348 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.207835 8123 tablet_service.cc:1505] Processing CreateTablet for tablet ebc0e6d526364d4ba37e443d1e681fbc (DEFAULT_TABLE table=default.Table_4f4721dca71f432b82e320f8fa9b0cf5 [id=bc3f005bc9424552831e2ecca5d049a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.207866 8616 consensus_queue.cc:1048] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.208078 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc0e6d526364d4ba37e443d1e681fbc. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:53.208187 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 02f6a32db68e4c609da39c3ee6c97348 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02f6a32db68e4c609da39c3ee6c97348 already in progress: creating tablet
I20251028 02:59:53.209990 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet d248313db35e40e38e2c5e07e68a69af with delete_type TABLET_DATA_DELETED (replaced by tablet d9c54e91d2d94573b40b50752318536d) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.210311 8550 tablet_replica.cc:333] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.210436 8550 raft_consensus.cc:2243] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.210546 8550 raft_consensus.cc:2272] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.210594 8912 consensus_queue.cc:1048] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.211550 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3410b1b517b146d4b38cb47862c85e68
I20251028 02:59:53.211843 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 3410b1b517b146d4b38cb47862c85e68 deleted and replaced by tablet ce64e4407ad048e8add422d77de6c211
I20251028 02:59:53.211982 8550 ts_tablet_manager.cc:1916] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.212304 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 0703e18d7dba4d628e3cc310be55fcf5 (table default.Table_8fdeffc6f6234305a829715139f150db [id=d4eaebd005bc42dd86d7125ef555546f]). Expected version 2 got 0
I20251028 02:59:53.213374 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 3410b1b517b146d4b38cb47862c85e68 with delete_type TABLET_DATA_DELETED (replaced by tablet ce64e4407ad048e8add422d77de6c211) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.213372 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 02f6a32db68e4c609da39c3ee6c97348 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.214219 8550 ts_tablet_manager.cc:1929] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.214303 8550 log.cc:1199] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/d248313db35e40e38e2c5e07e68a69af
I20251028 02:59:53.214363 8551 tablet_replica.cc:333] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.214529 8551 raft_consensus.cc:2243] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.214568 8550 ts_tablet_manager.cc:1950] T d248313db35e40e38e2c5e07e68a69af P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.214655 8551 raft_consensus.cc:2272] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.215364 8551 ts_tablet_manager.cc:1916] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.216262 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 3410b1b517b146d4b38cb47862c85e68 with delete_type TABLET_DATA_DELETED (replaced by tablet ce64e4407ad048e8add422d77de6c211) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.216336 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5fd9cc1847ff4df7a09e2e698786d515
I20251028 02:59:53.216598 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 83495009a1dc4f4388d2ae65f2dda548 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.217746 8551 ts_tablet_manager.cc:1929] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.217828 8551 log.cc:1199] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3410b1b517b146d4b38cb47862c85e68
I20251028 02:59:53.218084 8551 ts_tablet_manager.cc:1950] T 3410b1b517b146d4b38cb47862c85e68 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.218370 8550 tablet_replica.cc:333] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.218508 8550 raft_consensus.cc:2243] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.218626 8550 raft_consensus.cc:2272] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.219192 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3410b1b517b146d4b38cb47862c85e68 (table default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]) successfully deleted
I20251028 02:59:53.219349 8550 ts_tablet_manager.cc:1916] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.219841 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 3410b1b517b146d4b38cb47862c85e68 with delete_type TABLET_DATA_DELETED (replaced by tablet ce64e4407ad048e8add422d77de6c211) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.220247 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fd9cc1847ff4df7a09e2e698786d515 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:53.220747 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 02f6a32db68e4c609da39c3ee6c97348 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02f6a32db68e4c609da39c3ee6c97348 already in progress: creating tablet
W20251028 02:59:53.221468 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 5fd9cc1847ff4df7a09e2e698786d515 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fd9cc1847ff4df7a09e2e698786d515 already in progress: creating tablet
I20251028 02:59:53.221633 8550 ts_tablet_manager.cc:1929] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.221710 8550 log.cc:1199] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3410b1b517b146d4b38cb47862c85e68
I20251028 02:59:53.222095 8550 ts_tablet_manager.cc:1950] T 3410b1b517b146d4b38cb47862c85e68 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.222693 8550 tablet_replica.cc:333] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.222812 8550 raft_consensus.cc:2243] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.222898 8550 raft_consensus.cc:2272] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.223544 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet e359f0121c334cc1a8b3eedbbe64bda2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.224239 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fd9cc1847ff4df7a09e2e698786d515 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.224344 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_cc38c53f9a064f3f8af6b4d688c1aada" } new_table_name: "default.Table_5f8a6ff79e8f4ad7aee8aa99b07f20ba" modify_external_catalogs: true
I20251028 02:59:53.224359 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_33d362b6df3e40b68a3221d89a6b8358"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.224896 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33d362b6df3e40b68a3221d89a6b8358 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.224742 8550 ts_tablet_manager.cc:1916] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.225179 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 2be7be7c732e4780859cbe6624edf54e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.225310 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d711941006d4417901135b7761a251f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.225436 8551 tablet_replica.cc:333] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.225540 8551 raft_consensus.cc:2243] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.225629 8551 raft_consensus.cc:2272] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.225561 7960 catalog_manager.cc:5649] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:53.225718 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b237c0d768794e3592deda86eb2a4945" } new_table_name: "default.Table_2050be1c77ea4de0a7e510783a5f4009" modify_external_catalogs: true
I20251028 02:59:53.226145 8551 ts_tablet_manager.cc:1916] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.226187 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet d248313db35e40e38e2c5e07e68a69af (table default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]) successfully deleted
I20251028 02:59:53.226328 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3410b1b517b146d4b38cb47862c85e68 (table default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]) successfully deleted
I20251028 02:59:53.225633 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.226836 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 3410b1b517b146d4b38cb47862c85e68 with delete_type TABLET_DATA_DELETED (replaced by tablet ce64e4407ad048e8add422d77de6c211) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.228070 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fd9cc1847ff4df7a09e2e698786d515 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.228428 8551 ts_tablet_manager.cc:1929] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.229136 8551 log.cc:1199] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/e359f0121c334cc1a8b3eedbbe64bda2
I20251028 02:59:53.229067 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_322bdb2bdb474e39937bd5b0ecf1556e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.229197 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0b623fec605f49f29902835efaa5e71b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.229348 8551 ts_tablet_manager.cc:1950] T e359f0121c334cc1a8b3eedbbe64bda2 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 02:59:53.229478 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b623fec605f49f29902835efaa5e71b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:53.229483 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_322bdb2bdb474e39937bd5b0ecf1556e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.229987 8551 tablet_replica.cc:333] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.230108 8551 raft_consensus.cc:2243] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.230119 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet e359f0121c334cc1a8b3eedbbe64bda2 (table default.Table_726e0c2d9d4044d8b79bb266e21a8d9c [id=6630b83265864f36ad74cac692f131e3]) successfully deleted
I20251028 02:59:53.230204 8551 raft_consensus.cc:2272] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.230397 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 127e08303c564348b14257ec243418f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.230372 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_52becc2f534b4a0ab52b932cf5d2aec0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.230572 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 0141ebd1ddad421bbb8eca7615e78351 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:53.230703 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52becc2f534b4a0ab52b932cf5d2aec0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.230824 8551 ts_tablet_manager.cc:1916] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.233758 8544 tablet_bootstrap.cc:492] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.233901 8544 ts_tablet_manager.cc:1403] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.000s
I20251028 02:59:53.234462 8544 raft_consensus.cc:359] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.234594 8544 raft_consensus.cc:385] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.234632 8544 raft_consensus.cc:740] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.234748 8544 consensus_queue.cc:260] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.235057 8544 ts_tablet_manager.cc:1434] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.235284 8544 tablet_bootstrap.cc:492] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.236368 8544 tablet_bootstrap.cc:654] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.236577 8912 raft_consensus.cc:493] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.236658 8912 raft_consensus.cc:515] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.237007 8912 leader_election.cc:290] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.238195 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488bc52f1219478ab8843c9b1d79de2b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.238348 8421 raft_consensus.cc:2393] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:53.238571 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "488bc52f1219478ab8843c9b1d79de2b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.239269 8551 ts_tablet_manager.cc:1929] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.239322 8083 leader_election.cc:304] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:53.239372 8551 log.cc:1199] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2be7be7c732e4780859cbe6624edf54e
I20251028 02:59:53.239408 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet e6f9b707ec0345249c3efec1c24a956e
I20251028 02:59:53.239477 8656 raft_consensus.cc:3060] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.239701 7966 catalog_manager.cc:6719] ReplaceTablet: tablet e6f9b707ec0345249c3efec1c24a956e deleted and replaced by tablet 1ef353f5e715452eae1f1c191fe79d1d
I20251028 02:59:53.239890 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_114c71e193e84c098496acdd966b7fb7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.240305 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_114c71e193e84c098496acdd966b7fb7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.240478 8656 raft_consensus.cc:2749] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.240541 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet e6f9b707ec0345249c3efec1c24a956e with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef353f5e715452eae1f1c191fe79d1d) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.241358 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet e6f9b707ec0345249c3efec1c24a956e with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef353f5e715452eae1f1c191fe79d1d) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.241864 8551 ts_tablet_manager.cc:1950] T 2be7be7c732e4780859cbe6624edf54e P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.242062 8545 tablet_bootstrap.cc:492] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.242148 8545 ts_tablet_manager.cc:1403] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.058s user 0.002s sys 0.000s
I20251028 02:59:53.242136 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet e6f9b707ec0345249c3efec1c24a956e with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef353f5e715452eae1f1c191fe79d1d) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.242558 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2be7be7c732e4780859cbe6624edf54e (table default.Table_2a09149ed84d4343a81458e2fb78f262 [id=2bcc83cc472b4ec5a7ca79a8a7c5ff23]) successfully deleted
I20251028 02:59:53.242622 8545 raft_consensus.cc:359] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.242707 8552 ts_tablet_manager.cc:1916] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.242735 8545 raft_consensus.cc:385] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.242769 8545 raft_consensus.cc:740] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.242949 8545 consensus_queue.cc:260] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.243220 8545 ts_tablet_manager.cc:1434] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.243420 8545 tablet_bootstrap.cc:492] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.244482 8545 tablet_bootstrap.cc:654] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.245162 8552 ts_tablet_manager.cc:1929] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.245237 8552 log.cc:1199] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/d248313db35e40e38e2c5e07e68a69af
I20251028 02:59:53.245498 8552 ts_tablet_manager.cc:1950] T d248313db35e40e38e2c5e07e68a69af P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.242769 8551 tablet_replica.cc:333] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.246167 8545 tablet_bootstrap.cc:492] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.246306 8545 ts_tablet_manager.cc:1403] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251028 02:59:53.246404 8552 tablet_replica.cc:333] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.246538 8552 raft_consensus.cc:2243] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:53.246750 8552 raft_consensus.cc:2272] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.247030 8545 raft_consensus.cc:359] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.247208 8545 raft_consensus.cc:385] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.247242 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet d248313db35e40e38e2c5e07e68a69af (table default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]) successfully deleted
I20251028 02:59:53.247292 8545 raft_consensus.cc:740] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
W20251028 02:59:53.247478 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 02f6a32db68e4c609da39c3ee6c97348 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02f6a32db68e4c609da39c3ee6c97348 already in progress: creating tablet
I20251028 02:59:53.247604 8545 consensus_queue.cc:260] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.247974 8545 ts_tablet_manager.cc:1434] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 02:59:53.248199 8545 tablet_bootstrap.cc:492] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.248752 8551 raft_consensus.cc:2243] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.248880 8551 raft_consensus.cc:2272] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.249090 8965 consensus_queue.cc:1048] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.250128 8544 tablet_bootstrap.cc:492] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.250231 8544 ts_tablet_manager.cc:1403] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20251028 02:59:53.250766 8544 raft_consensus.cc:359] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.250893 8544 raft_consensus.cc:385] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.250947 8544 raft_consensus.cc:740] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.250869 8545 tablet_bootstrap.cc:654] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.251089 8544 consensus_queue.cc:260] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.251432 8544 ts_tablet_manager.cc:1434] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:53.251503 8552 ts_tablet_manager.cc:1916] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.251647 8544 tablet_bootstrap.cc:492] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.252652 8544 tablet_bootstrap.cc:654] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:53.253052 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ec69044a4c3e4f049f90be835cf73367 consensus state has no leader
I20251028 02:59:53.254240 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 3057f4749edd43ac889f64d1f4e282d1 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.254603 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7e5a0a5655db40d8957f85af13a091b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.254983 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e5a0a5655db40d8957f85af13a091b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.255856 8551 ts_tablet_manager.cc:1916] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.256057 7786 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu with pid 8559
I20251028 02:59:53.257406 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c6ab20b423864891ba7a399af893a205"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.257795 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6ab20b423864891ba7a399af893a205 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.259642 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 99968de685b44cbfa4c002a6c40e9cb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e9234b72f4448089fe7ff6ba49483b7) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.259822 8556 consensus_queue.cc:1048] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.261252 7961 catalog_manager.cc:5649] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:53.261584 7961 catalog_manager.cc:5649] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:53.261792 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 9c1d8552cc1941bda2e9f02fba56c9f9 (table default.Table_bb551b322c5140ba89d154ff450085dd [id=4e8c5072da4e4ec6a616ba4c7fc91cd3]). Expected version 1 got 0
I20251028 02:59:53.262459 8545 tablet_bootstrap.cc:492] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.262564 8545 ts_tablet_manager.cc:1403] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20251028 02:59:53.262863 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 76f4055230904a25a1241a0a86a059d2
I20251028 02:59:53.263108 8545 raft_consensus.cc:359] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.263213 8545 raft_consensus.cc:385] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.263250 8545 raft_consensus.cc:740] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.263435 8545 consensus_queue.cc:260] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.263751 8545 ts_tablet_manager.cc:1434] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.263952 8545 tablet_bootstrap.cc:492] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.264035 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 2be7be7c732e4780859cbe6624edf54e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.264652 8552 ts_tablet_manager.cc:1929] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.264740 8552 log.cc:1199] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3410b1b517b146d4b38cb47862c85e68
I20251028 02:59:53.266100 8552 ts_tablet_manager.cc:1950] T 3410b1b517b146d4b38cb47862c85e68 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.266252 8545 tablet_bootstrap.cc:654] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.266320 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 76f4055230904a25a1241a0a86a059d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.267063 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 76f4055230904a25a1241a0a86a059d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.267217 8551 ts_tablet_manager.cc:1929] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.267315 8551 log.cc:1199] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2d711941006d4417901135b7761a251f
I20251028 02:59:53.267539 8551 ts_tablet_manager.cc:1950] T 2d711941006d4417901135b7761a251f P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.267722 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3410b1b517b146d4b38cb47862c85e68 (table default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]) successfully deleted
I20251028 02:59:53.268188 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2d711941006d4417901135b7761a251f (table default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]) successfully deleted
I20251028 02:59:53.268234 8551 tablet_replica.cc:333] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.268293 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 76f4055230904a25a1241a0a86a059d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:53.268350 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 5fd9cc1847ff4df7a09e2e698786d515 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fd9cc1847ff4df7a09e2e698786d515 already in progress: creating tablet
I20251028 02:59:53.268385 8551 raft_consensus.cc:2243] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:53.268476 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet e6f9b707ec0345249c3efec1c24a956e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6f9b707ec0345249c3efec1c24a956e already in progress: creating tablet
I20251028 02:59:53.268486 8551 raft_consensus.cc:2272] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:53.268599 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 76f4055230904a25a1241a0a86a059d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76f4055230904a25a1241a0a86a059d2 already in progress: creating tablet
I20251028 02:59:53.269032 8551 ts_tablet_manager.cc:1916] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.269274 8550 ts_tablet_manager.cc:1929] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.269351 8550 log.cc:1199] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/83495009a1dc4f4388d2ae65f2dda548
I20251028 02:59:53.269542 8550 ts_tablet_manager.cc:1950] T 83495009a1dc4f4388d2ae65f2dda548 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.270351 8550 tablet_replica.cc:333] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.270395 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 83495009a1dc4f4388d2ae65f2dda548 (table default.Table_309f970a07c6425bb6e764dfb593bc80 [id=27a892441799474b9572251752e147f3]) successfully deleted
W20251028 02:59:53.270591 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 5fd9cc1847ff4df7a09e2e698786d515 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fd9cc1847ff4df7a09e2e698786d515 already in progress: creating tablet
I20251028 02:59:53.270612 8550 raft_consensus.cc:2243] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.270848 8550 raft_consensus.cc:2272] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.271391 8550 ts_tablet_manager.cc:1916] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.271930 9048 tablet.cc:1722] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.272753 8551 ts_tablet_manager.cc:1929] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.272828 8551 log.cc:1199] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2bcbf3c72bb34c6291f4c1b1b9b1e906
I20251028 02:59:53.272931 9023 tablet.cc:1722] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.273039 8551 ts_tablet_manager.cc:1950] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.273695 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 (table default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]) successfully deleted
I20251028 02:59:53.273713 8550 ts_tablet_manager.cc:1929] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.273864 8550 log.cc:1199] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/127e08303c564348b14257ec243418f4
I20251028 02:59:53.274076 8550 ts_tablet_manager.cc:1950] T 127e08303c564348b14257ec243418f4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 02:59:53.274155 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 3410b1b517b146d4b38cb47862c85e68 because the tablet was not found. No further retry: Not found: Tablet not found: 3410b1b517b146d4b38cb47862c85e68
I20251028 02:59:53.274580 8550 tablet_replica.cc:333] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.274715 8550 raft_consensus.cc:2243] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:53.274822 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet e6f9b707ec0345249c3efec1c24a956e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6f9b707ec0345249c3efec1c24a956e already in progress: creating tablet
I20251028 02:59:53.274870 8550 raft_consensus.cc:2272] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.275131 8551 tablet_replica.cc:333] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.275668 8551 raft_consensus.cc:2243] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.275770 8551 raft_consensus.cc:2272] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.276126 8884 tablet.cc:1722] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.276572 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 127e08303c564348b14257ec243418f4 (table default.Table_24016eb841cb46299d394b16f2d4f3b8 [id=72061aad76324293b5485166aac98a4a]) successfully deleted
I20251028 02:59:53.278339 8550 ts_tablet_manager.cc:1916] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.278475 8551 ts_tablet_manager.cc:1916] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.280421 8544 tablet_bootstrap.cc:492] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.280514 8544 ts_tablet_manager.cc:1403] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 02:59:53.280789 8550 ts_tablet_manager.cc:1929] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.280864 8550 log.cc:1199] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/0141ebd1ddad421bbb8eca7615e78351
I20251028 02:59:53.281100 8550 ts_tablet_manager.cc:1950] T 0141ebd1ddad421bbb8eca7615e78351 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.281071 8544 raft_consensus.cc:359] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.281176 8544 raft_consensus.cc:385] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.281208 8544 raft_consensus.cc:740] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.281322 8544 consensus_queue.cc:260] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.281512 8551 ts_tablet_manager.cc:1929] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.281580 8551 log.cc:1199] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/76f4055230904a25a1241a0a86a059d2
I20251028 02:59:53.281586 8544 ts_tablet_manager.cc:1434] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.281785 8551 ts_tablet_manager.cc:1950] T 76f4055230904a25a1241a0a86a059d2 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.282742 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 76f4055230904a25a1241a0a86a059d2 (table default.Table_d2e5ac67a4414c03b7890dea514caeee [id=c8f1a159643642eb8dc2f202d73532fe]) successfully deleted
I20251028 02:59:53.284225 8549 consensus_queue.cc:1048] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.284543 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d012c9d921f542c4ab7a5e8858d351d4" } new_table_name: "default.Table_81a46304107f49a7ba522543a0659d5b" modify_external_catalogs: true
W20251028 02:59:53.285480 8218 connection.cc:537] client connection to 127.7.154.189:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251028 02:59:53.285710 7953 connection.cc:537] client connection to 127.7.154.189:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251028 02:59:53.285934 8562 consensus_queue.cc:1048] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:53.286494 8354 connection.cc:537] client connection to 127.7.154.189:45473 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251028 02:59:53.289225 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_63b74aebe1364823bb6a10f4a8942db8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.289639 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_63b74aebe1364823bb6a10f4a8942db8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:53.291177 9090 negotiation.cc:337] Failed RPC negotiation. Trace:
1028 02:59:53.286858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.154.189:45473 (local address 127.7.154.129:42129)
1028 02:59:53.290141 (+ 3283us) negotiation.cc:107] Waiting for socket to connect
1028 02:59:53.290208 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":1567,"thread_start_us":67,"threads_started":1}
I20251028 02:59:53.292181 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 0141ebd1ddad421bbb8eca7615e78351 (table default.Table_83adc71d84344ff686fc777f2f72d54b [id=0b22f43d0bda44b98e6f4b2f26153c5c]) successfully deleted
W20251028 02:59:53.292559 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e6f9b707ec0345249c3efec1c24a956e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6f9b707ec0345249c3efec1c24a956e already in progress: creating tablet
I20251028 02:59:53.292732 8550 tablet_replica.cc:333] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.292855 8550 raft_consensus.cc:2243] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.292937 8550 raft_consensus.cc:2272] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:53.293260 9089 negotiation.cc:337] Failed RPC negotiation. Trace:
1028 02:59:53.286156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.154.189:45473 (local address 127.7.154.130:48795)
1028 02:59:53.292594 (+ 6438us) negotiation.cc:107] Waiting for socket to connect
1028 02:59:53.292676 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":6347,"thread_start_us":65,"threads_started":1}
W20251028 02:59:53.293668 9092 negotiation.cc:337] Failed RPC negotiation. Trace:
1028 02:59:53.288065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.154.189:45473 (local address 127.7.154.131:46995)
1028 02:59:53.293149 (+ 5084us) negotiation.cc:107] Waiting for socket to connect
1028 02:59:53.293206 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":5003,"thread_start_us":58,"threads_started":1}
W20251028 02:59:53.295214 7955 catalog_manager.cc:5058] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): alter failed for tablet 71feaa313a8f45798561741f7b5d9b20 (table default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]),no further retry: Not found: Tablet not found: 71feaa313a8f45798561741f7b5d9b20
I20251028 02:59:53.295676 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6b3cfd5f43ca4f3b82a37dab711251b0" } new_table_name: "default.Table_bb39271bc8354db88ef788080eb177bc" modify_external_catalogs: true
I20251028 02:59:53.297365 8544 tablet_bootstrap.cc:492] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.298399 8544 tablet_bootstrap.cc:654] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.299602 8544 tablet_bootstrap.cc:492] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.299678 8550 ts_tablet_manager.cc:1916] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.299695 8544 ts_tablet_manager.cc:1403] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20251028 02:59:53.300266 8544 raft_consensus.cc:359] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.300383 8544 raft_consensus.cc:385] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.300415 8544 raft_consensus.cc:740] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.300568 8544 consensus_queue.cc:260] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.300932 8544 ts_tablet_manager.cc:1434] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:53.301138 8544 tablet_bootstrap.cc:492] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.302074 8550 ts_tablet_manager.cc:1929] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.302153 8550 log.cc:1199] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/99968de685b44cbfa4c002a6c40e9cb3
I20251028 02:59:53.302392 8550 ts_tablet_manager.cc:1950] T 99968de685b44cbfa4c002a6c40e9cb3 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.303006 8550 tablet_replica.cc:333] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.303129 8550 raft_consensus.cc:2243] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.303124 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 99968de685b44cbfa4c002a6c40e9cb3 (table default.Table_3d8776d793a54bd795fb38bd45442f51 [id=be1f381c3ca048d891f5d175fa72fc65]) successfully deleted
I20251028 02:59:53.303241 8550 raft_consensus.cc:2272] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.305084 8544 tablet_bootstrap.cc:654] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.306329 8546 tablet_bootstrap.cc:492] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.306424 8546 ts_tablet_manager.cc:1403] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.177s user 0.002s sys 0.000s
I20251028 02:59:53.306944 8546 raft_consensus.cc:359] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.307081 8546 raft_consensus.cc:385] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.307114 8546 raft_consensus.cc:740] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.307240 8546 consensus_queue.cc:260] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.307579 8546 ts_tablet_manager.cc:1434] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.307792 8546 tablet_bootstrap.cc:492] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:53.308581 7953 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d -> Peer aac264d6d2274d3283bd3251ab77e583 (127.7.154.189:45473): Couldn't send request to peer aac264d6d2274d3283bd3251ab77e583. Status: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:53.309841 8546 tablet_bootstrap.cc:654] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.310930 8546 tablet_bootstrap.cc:492] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.311023 8546 ts_tablet_manager.cc:1403] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:53.311105 8545 tablet_bootstrap.cc:492] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.311194 8545 ts_tablet_manager.cc:1403] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20251028 02:59:53.311841 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8bd9c3af53644cbcb68461ae9433a9f1" } modify_external_catalogs: true
I20251028 02:59:53.311964 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8bd9c3af53644cbcb68461ae9433a9f1" } modify_external_catalogs: true
I20251028 02:59:53.312346 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_8cae8460736e4dcb9108757c62255907"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.312734 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8cae8460736e4dcb9108757c62255907 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.315341 8545 raft_consensus.cc:359] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.315481 8545 raft_consensus.cc:385] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.315518 8545 raft_consensus.cc:740] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.315654 8545 consensus_queue.cc:260] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.316202 8545 ts_tablet_manager.cc:1434] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 02:59:53.316431 8545 tablet_bootstrap.cc:492] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.316609 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 617ca0f545824d14ba7270619f121bcc from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.317471 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d0d760998d424df696eb71d3f8223366"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.317643 8545 tablet_bootstrap.cc:654] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.317812 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 552c509b32374f5f9e3c232fbeddf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.317876 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 76f4055230904a25a1241a0a86a059d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:53.317885 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0d760998d424df696eb71d3f8223366 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.317987 8551 tablet_replica.cc:333] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.318130 8551 raft_consensus.cc:2243] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.318235 8551 raft_consensus.cc:2272] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:53.318316 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 37dcf605a2984be1817d9621e4a41448 consensus state has no leader
I20251028 02:59:53.318509 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d] alter complete (version 3)
I20251028 02:59:53.318717 8551 ts_tablet_manager.cc:1916] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:53.318894 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df10bbbd757a4da8905718b61cbb10c8 consensus state has no leader
I20251028 02:59:53.318976 8550 ts_tablet_manager.cc:1916] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.321880 8551 ts_tablet_manager.cc:1929] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.321974 8551 log.cc:1199] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/552c509b32374f5f9e3c232fbeddf9bd
I20251028 02:59:53.322225 8551 ts_tablet_manager.cc:1950] T 552c509b32374f5f9e3c232fbeddf9bd P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.322655 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0b623fec605f49f29902835efaa5e71b" } modify_external_catalogs: true
I20251028 02:59:53.322769 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0b623fec605f49f29902835efaa5e71b" } modify_external_catalogs: true
I20251028 02:59:53.323606 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 552c509b32374f5f9e3c232fbeddf9bd (table default.Table_e3e2db3778b049ce8d0327aa81f7e2ae [id=83c18d626c264667b358e53e0b6fda63]) successfully deleted
W20251028 02:59:53.324062 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 76f4055230904a25a1241a0a86a059d2 because the tablet was not found. No further retry: Not found: Tablet not found: 76f4055230904a25a1241a0a86a059d2
I20251028 02:59:53.324214 8550 ts_tablet_manager.cc:1929] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.324282 8550 log.cc:1199] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2be7be7c732e4780859cbe6624edf54e
I20251028 02:59:53.324489 8550 ts_tablet_manager.cc:1950] T 2be7be7c732e4780859cbe6624edf54e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.325169 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2be7be7c732e4780859cbe6624edf54e (table default.Table_2a09149ed84d4343a81458e2fb78f262 [id=2bcc83cc472b4ec5a7ca79a8a7c5ff23]) successfully deleted
I20251028 02:59:53.326646 8421 raft_consensus.cc:1275] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:53.327095 8553 consensus_queue.cc:1048] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.327183 8148 raft_consensus.cc:3060] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:53.327412 7953 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 617ca0f545824d14ba7270619f121bcc (table default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 617ca0f545824d14ba7270619f121bcc consensus state has no leader
I20251028 02:59:53.328279 8148 raft_consensus.cc:1275] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:53.328369 8545 tablet_bootstrap.cc:492] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.328456 8545 ts_tablet_manager.cc:1403] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:53.329080 8545 raft_consensus.cc:359] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.329139 8553 consensus_queue.cc:1048] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.329186 8545 raft_consensus.cc:385] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.329893 8884 tablet.cc:1722] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:53.330821 9048 tablet.cc:1722] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:53.331097 8546 raft_consensus.cc:359] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.329277 8545 raft_consensus.cc:740] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.331506 8545 consensus_queue.cc:260] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.331884 8545 ts_tablet_manager.cc:1434] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:53.331192 8546 raft_consensus.cc:385] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.332000 8546 raft_consensus.cc:740] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.332135 8546 consensus_queue.cc:260] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.332509 8546 ts_tablet_manager.cc:1434] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.021s user 0.001s sys 0.000s
I20251028 02:59:53.332695 8546 tablet_bootstrap.cc:492] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.334090 8546 tablet_bootstrap.cc:654] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.335160 8545 tablet_bootstrap.cc:492] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.336557 8545 tablet_bootstrap.cc:654] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.337695 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8cb97d5192684d748b8b485f891b0dc0
I20251028 02:59:53.338320 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 8cb97d5192684d748b8b485f891b0dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.338882 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 8cb97d5192684d748b8b485f891b0dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.338963 8262 tablet_service.cc:1552] Processing DeleteTablet for tablet 8cb97d5192684d748b8b485f891b0dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.339066 8551 tablet_replica.cc:333] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.339174 8551 raft_consensus.cc:2243] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.339262 8551 raft_consensus.cc:2272] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.340680 8551 ts_tablet_manager.cc:1916] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.342895 8551 ts_tablet_manager.cc:1929] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:53.342978 8551 log.cc:1199] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8cb97d5192684d748b8b485f891b0dc0
I20251028 02:59:53.343228 8551 ts_tablet_manager.cc:1950] T 8cb97d5192684d748b8b485f891b0dc0 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.343467 8552 tablet_replica.cc:333] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.343629 8552 raft_consensus.cc:2243] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:53.343838 8552 raft_consensus.cc:2272] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.344345 8552 ts_tablet_manager.cc:1916] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.345206 7962 catalog_manager.cc:5931] Not sending DeleteTablet requests; no consensus state for tablet 246f50b537d44e699c374b4a3e298eb0
I20251028 02:59:53.347815 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]). Expected version 1 got 0
I20251028 02:59:53.349150 9023 tablet.cc:1722] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20251028 02:59:53.349493 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 76f4055230904a25a1241a0a86a059d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 76f4055230904a25a1241a0a86a059d2 already in progress: creating tablet
I20251028 02:59:53.349771 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 9409f3ee379949809f823dfb7f73643b (DEFAULT_TABLE table=default.Table_0a1e2cd2aace496790ca53a8bdfbea25 [id=cf6c49c2909c46ce97c6f51a2e7f5300]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.349830 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 19efc09c08b540a3b42c4107de5acf73 (DEFAULT_TABLE table=default.Table_4b9a111a46cd448cac4abf954888e213 [id=097df645a88549b68a7c84c94f158190]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.350040 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9409f3ee379949809f823dfb7f73643b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.350468 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 4bcdf0368e394d649086042f1fe9774b (DEFAULT_TABLE table=default.Table_2d9658afb66c44639064e40eaf4fa075 [id=575b84e0d3b743a0806edaae38121f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.350692 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bcdf0368e394d649086042f1fe9774b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.351111 8552 ts_tablet_manager.cc:1929] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:53.351181 8552 log.cc:1199] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/8cb97d5192684d748b8b485f891b0dc0
I20251028 02:59:53.352448 8552 ts_tablet_manager.cc:1950] T 8cb97d5192684d748b8b485f891b0dc0 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.353183 8117 tablet_service.cc:1505] Processing CreateTablet for tablet d9c54e91d2d94573b40b50752318536d (DEFAULT_TABLE table=default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.353401 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c54e91d2d94573b40b50752318536d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.353812 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 4246c4bd2abf4569987ca346996445d4 (DEFAULT_TABLE table=default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.354018 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4246c4bd2abf4569987ca346996445d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.354065 8545 tablet_bootstrap.cc:492] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.354156 8545 ts_tablet_manager.cc:1403] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20251028 02:59:53.354255 8546 tablet_bootstrap.cc:492] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.354341 8546 ts_tablet_manager.cc:1403] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20251028 02:59:53.354707 8545 raft_consensus.cc:359] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.354827 8545 raft_consensus.cc:385] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.354861 8545 raft_consensus.cc:740] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.354825 8546 raft_consensus.cc:359] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.354941 8546 raft_consensus.cc:385] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.354976 8546 raft_consensus.cc:740] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.355016 8545 consensus_queue.cc:260] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.355105 8546 consensus_queue.cc:260] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.355427 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19efc09c08b540a3b42c4107de5acf73. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.355551 8545 ts_tablet_manager.cc:1434] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:53.355737 8545 tablet_bootstrap.cc:492] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.356768 8545 tablet_bootstrap.cc:654] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.357170 8546 ts_tablet_manager.cc:1434] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:53.357355 8546 tablet_bootstrap.cc:492] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.358387 8546 tablet_bootstrap.cc:654] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.358584 8549 raft_consensus.cc:493] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.358666 8549 raft_consensus.cc:515] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.358781 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 8cb97d5192684d748b8b485f891b0dc0 (table default.Table_8bd9c3af53644cbcb68461ae9433a9f1 [id=a9662d8045574f0abf81c9e6bc2f180c]) successfully deleted
I20251028 02:59:53.359076 8549 leader_election.cc:290] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.359283 8262 tablet_service.cc:1505] Processing CreateTablet for tablet 4bcdf0368e394d649086042f1fe9774b (DEFAULT_TABLE table=default.Table_2d9658afb66c44639064e40eaf4fa075 [id=575b84e0d3b743a0806edaae38121f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.359462 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 4246c4bd2abf4569987ca346996445d4 (DEFAULT_TABLE table=default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.359597 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bcdf0368e394d649086042f1fe9774b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.359639 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 9409f3ee379949809f823dfb7f73643b (DEFAULT_TABLE table=default.Table_0a1e2cd2aace496790ca53a8bdfbea25 [id=cf6c49c2909c46ce97c6f51a2e7f5300]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.359886 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9409f3ee379949809f823dfb7f73643b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.360265 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 19efc09c08b540a3b42c4107de5acf73 (DEFAULT_TABLE table=default.Table_4b9a111a46cd448cac4abf954888e213 [id=097df645a88549b68a7c84c94f158190]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.360349 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet db6eadc5927d4876af9a50b44be97a63 with delete_type TABLET_DATA_DELETED (replaced by tablet 67c6ceefa3294af8b68150609f46c117) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.360484 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19efc09c08b540a3b42c4107de5acf73. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.360622 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 9409f3ee379949809f823dfb7f73643b (DEFAULT_TABLE table=default.Table_0a1e2cd2aace496790ca53a8bdfbea25 [id=cf6c49c2909c46ce97c6f51a2e7f5300]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.360862 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9409f3ee379949809f823dfb7f73643b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.361886 8260 tablet_service.cc:1505] Processing CreateTablet for tablet d9c54e91d2d94573b40b50752318536d (DEFAULT_TABLE table=default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.362120 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c54e91d2d94573b40b50752318536d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.362739 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet aeddf70e5bd342f19f3f427d1df73814 with delete_type TABLET_DATA_DELETED (replaced by tablet 5fd9cc1847ff4df7a09e2e698786d515) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.362717 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 4bcdf0368e394d649086042f1fe9774b (DEFAULT_TABLE table=default.Table_2d9658afb66c44639064e40eaf4fa075 [id=575b84e0d3b743a0806edaae38121f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.362946 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 4246c4bd2abf4569987ca346996445d4 (DEFAULT_TABLE table=default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.363042 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bcdf0368e394d649086042f1fe9774b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.363166 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4246c4bd2abf4569987ca346996445d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.363430 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 617ca0f545824d14ba7270619f121bcc
I20251028 02:59:53.363698 7964 catalog_manager.cc:6719] ReplaceTablet: tablet 617ca0f545824d14ba7270619f121bcc deleted and replaced by tablet 73c20437695a4043a86faa9c98091e5e
I20251028 02:59:53.363970 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3057f4749edd43ac889f64d1f4e282d1
I20251028 02:59:53.364132 8551 tablet_replica.cc:333] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.364226 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 3057f4749edd43ac889f64d1f4e282d1 deleted and replaced by tablet 2b5908fd54944004866ca3864e8131d2
I20251028 02:59:53.364275 8551 raft_consensus.cc:2243] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.364393 8551 raft_consensus.cc:2272] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.364409 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "df10bbbd757a4da8905718b61cbb10c8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.359283 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 19efc09c08b540a3b42c4107de5acf73 (DEFAULT_TABLE table=default.Table_4b9a111a46cd448cac4abf954888e213 [id=097df645a88549b68a7c84c94f158190]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.364851 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19efc09c08b540a3b42c4107de5acf73. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.365238 8397 tablet_service.cc:1505] Processing CreateTablet for tablet d9c54e91d2d94573b40b50752318536d (DEFAULT_TABLE table=default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.365442 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c54e91d2d94573b40b50752318536d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.367056 8551 ts_tablet_manager.cc:1916] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.367728 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8cb97d5192684d748b8b485f891b0dc0 (table default.Table_8bd9c3af53644cbcb68461ae9433a9f1 [id=a9662d8045574f0abf81c9e6bc2f180c]) successfully deleted
I20251028 02:59:53.368357 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4246c4bd2abf4569987ca346996445d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.368950 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 552c509b32374f5f9e3c232fbeddf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.369102 8551 ts_tablet_manager.cc:1929] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.369179 8551 log.cc:1199] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/db6eadc5927d4876af9a50b44be97a63
I20251028 02:59:53.369400 8551 ts_tablet_manager.cc:1950] T db6eadc5927d4876af9a50b44be97a63 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.369436 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet df10bbbd757a4da8905718b61cbb10c8 (table default.Table_bb39271bc8354db88ef788080eb177bc [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 3 got 0
I20251028 02:59:53.369556 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet da1563d65b4d4ab48325a4f08ee89ed5 (table default.Table_153d49d056a3458a9565c70d954187ae [id=ae8a178b437f4df4af15bb82bd7a95ee]). Expected version 1 got 0
I20251028 02:59:53.369978 8551 tablet_replica.cc:333] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.370132 8551 raft_consensus.cc:2243] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.370229 8551 raft_consensus.cc:2272] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.371417 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 617ca0f545824d14ba7270619f121bcc with delete_type TABLET_DATA_DELETED (replaced by tablet 73c20437695a4043a86faa9c98091e5e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.371621 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 3057f4749edd43ac889f64d1f4e282d1 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b5908fd54944004866ca3864e8131d2) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.371958 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "df10bbbd757a4da8905718b61cbb10c8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.371870 8550 tablet_replica.cc:333] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.372239 8551 ts_tablet_manager.cc:1916] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.372278 8550 raft_consensus.cc:2243] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.372485 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 617ca0f545824d14ba7270619f121bcc with delete_type TABLET_DATA_DELETED (replaced by tablet 73c20437695a4043a86faa9c98091e5e) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.372395 8550 raft_consensus.cc:2272] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:53.372586 8356 leader_election.cc:343] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.373093 8550 ts_tablet_manager.cc:1916] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:53.373186 8354 leader_election.cc:343] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.373243 8354 leader_election.cc:304] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.373436 8549 raft_consensus.cc:2749] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.373745 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 3057f4749edd43ac889f64d1f4e282d1 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b5908fd54944004866ca3864e8131d2) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.374097 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 3057f4749edd43ac889f64d1f4e282d1 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b5908fd54944004866ca3864e8131d2) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.374356 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet db6eadc5927d4876af9a50b44be97a63 (table default.Table_12396134528e4544a80934aa91d07353 [id=86c2f59df3dd4d2195d18835dceff79b]) successfully deleted
I20251028 02:59:53.374543 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.375414 8552 tablet_replica.cc:333] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.375547 8550 ts_tablet_manager.cc:1929] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:53.375620 8550 log.cc:1199] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8cb97d5192684d748b8b485f891b0dc0
I20251028 02:59:53.375643 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 617ca0f545824d14ba7270619f121bcc with delete_type TABLET_DATA_DELETED (replaced by tablet 73c20437695a4043a86faa9c98091e5e) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.375690 8549 raft_consensus.cc:493] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.375768 8549 raft_consensus.cc:515] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.375895 8550 ts_tablet_manager.cc:1950] T 8cb97d5192684d748b8b485f891b0dc0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.376183 8549 leader_election.cc:290] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.376533 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8cb97d5192684d748b8b485f891b0dc0 (table default.Table_8bd9c3af53644cbcb68461ae9433a9f1 [id=a9662d8045574f0abf81c9e6bc2f180c]) successfully deleted
I20251028 02:59:53.377107 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c048520fe7b248e586e44a2047ffb239" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.377384 8550 tablet_replica.cc:333] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.377532 8550 raft_consensus.cc:2243] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.377650 8550 raft_consensus.cc:2272] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:53.377642 8354 leader_election.cc:343] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.378255 8551 ts_tablet_manager.cc:1929] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.378372 8551 log.cc:1199] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/aeddf70e5bd342f19f3f427d1df73814
I20251028 02:59:53.378829 8550 ts_tablet_manager.cc:1916] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.379259 8551 ts_tablet_manager.cc:1950] T aeddf70e5bd342f19f3f427d1df73814 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.376930 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c048520fe7b248e586e44a2047ffb239" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.380167 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet aeddf70e5bd342f19f3f427d1df73814 (table default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]) successfully deleted
I20251028 02:59:53.380205 8544 tablet_bootstrap.cc:492] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.379640 8546 tablet_bootstrap.cc:492] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.380280 8544 ts_tablet_manager.cc:1403] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.079s user 0.000s sys 0.002s
I20251028 02:59:53.380313 8546 ts_tablet_manager.cc:1403] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.001s
I20251028 02:59:53.380481 8552 raft_consensus.cc:2243] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.380585 8552 raft_consensus.cc:2272] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:53.380609 8356 leader_election.cc:343] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.380676 8356 leader_election.cc:304] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.380780 8544 raft_consensus.cc:359] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.380827 8546 raft_consensus.cc:359] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.380898 8544 raft_consensus.cc:385] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.380887 8549 raft_consensus.cc:2749] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.380928 8546 raft_consensus.cc:385] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.380934 8544 raft_consensus.cc:740] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.380961 8546 raft_consensus.cc:740] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.381064 8544 consensus_queue.cc:260] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.381093 8546 consensus_queue.cc:260] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.381345 8544 ts_tablet_manager.cc:1434] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251028 02:59:53.381374 8546 ts_tablet_manager.cc:1434] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:53.381532 8544 tablet_bootstrap.cc:492] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.381558 8546 tablet_bootstrap.cc:492] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.382539 8552 ts_tablet_manager.cc:1916] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.382584 8544 tablet_bootstrap.cc:654] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:53.382851 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 617ca0f545824d14ba7270619f121bcc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 617ca0f545824d14ba7270619f121bcc already in progress: creating tablet
I20251028 02:59:53.383147 8551 tablet_replica.cc:333] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.383329 8551 raft_consensus.cc:2243] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.383589 8551 raft_consensus.cc:2272] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.383874 8546 tablet_bootstrap.cc:654] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.384163 8551 ts_tablet_manager.cc:1916] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.384676 8545 tablet_bootstrap.cc:492] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.384778 8545 ts_tablet_manager.cc:1403] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 02:59:53.385313 8545 raft_consensus.cc:359] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.385417 8545 raft_consensus.cc:385] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.385455 8545 raft_consensus.cc:740] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.385620 8545 consensus_queue.cc:260] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.386907 8545 ts_tablet_manager.cc:1434] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:53.386991 8550 ts_tablet_manager.cc:1929] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.387079 8550 log.cc:1199] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3057f4749edd43ac889f64d1f4e282d1
I20251028 02:59:53.387120 8545 tablet_bootstrap.cc:492] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.387354 8550 ts_tablet_manager.cc:1950] T 3057f4749edd43ac889f64d1f4e282d1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.388044 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3057f4749edd43ac889f64d1f4e282d1 (table default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]) successfully deleted
I20251028 02:59:53.388295 8550 tablet_replica.cc:333] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.388442 8545 tablet_bootstrap.cc:654] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.388435 8550 raft_consensus.cc:2243] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.388548 8550 raft_consensus.cc:2272] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.389600 8545 tablet_bootstrap.cc:492] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.389698 8545 ts_tablet_manager.cc:1403] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:53.389791 8550 ts_tablet_manager.cc:1916] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.390242 8545 raft_consensus.cc:359] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.390350 8545 raft_consensus.cc:385] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.390388 8545 raft_consensus.cc:740] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.386415 8551 ts_tablet_manager.cc:1929] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.390470 8551 log.cc:1199] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3057f4749edd43ac889f64d1f4e282d1
I20251028 02:59:53.390676 8552 ts_tablet_manager.cc:1929] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.390810 8552 log.cc:1199] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/552c509b32374f5f9e3c232fbeddf9bd
I20251028 02:59:53.390906 8551 ts_tablet_manager.cc:1950] T 3057f4749edd43ac889f64d1f4e282d1 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.391119 8552 ts_tablet_manager.cc:1950] T 552c509b32374f5f9e3c232fbeddf9bd P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.391534 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3057f4749edd43ac889f64d1f4e282d1 (table default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]) successfully deleted
I20251028 02:59:53.390522 8545 consensus_queue.cc:260] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.391959 8545 ts_tablet_manager.cc:1434] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:53.392079 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 552c509b32374f5f9e3c232fbeddf9bd (table default.Table_e3e2db3778b049ce8d0327aa81f7e2ae [id=83c18d626c264667b358e53e0b6fda63]) successfully deleted
I20251028 02:59:53.392182 8545 tablet_bootstrap.cc:492] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.392270 8544 tablet_bootstrap.cc:492] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.392366 8544 ts_tablet_manager.cc:1403] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
W20251028 02:59:53.392694 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 617ca0f545824d14ba7270619f121bcc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 617ca0f545824d14ba7270619f121bcc already in progress: creating tablet
I20251028 02:59:53.392856 8544 raft_consensus.cc:359] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.392971 8544 raft_consensus.cc:385] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.393018 8544 raft_consensus.cc:740] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.393136 8552 tablet_replica.cc:333] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.393141 8544 consensus_queue.cc:260] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.393211 8545 tablet_bootstrap.cc:654] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.393381 8552 raft_consensus.cc:2243] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:53.393412 8544 ts_tablet_manager.cc:1434] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:53.393597 8544 tablet_bootstrap.cc:492] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.393599 8552 pending_rounds.cc:70] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:53.393702 8552 pending_rounds.cc:77] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597130210607104 op_type: NO_OP noop_request { }
I20251028 02:59:53.393841 8552 raft_consensus.cc:2889] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:53.393965 8552 raft_consensus.cc:2272] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.394605 8544 tablet_bootstrap.cc:654] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.395561 8545 tablet_bootstrap.cc:492] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.395644 8545 ts_tablet_manager.cc:1403] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:53.396049 8545 raft_consensus.cc:359] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.396165 8545 raft_consensus.cc:385] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.396203 8545 raft_consensus.cc:740] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.396338 8546 tablet_bootstrap.cc:492] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.396320 8545 consensus_queue.cc:260] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.396426 8546 ts_tablet_manager.cc:1403] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20251028 02:59:53.396576 8545 ts_tablet_manager.cc:1434] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.396751 8545 tablet_bootstrap.cc:492] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.396946 8546 raft_consensus.cc:359] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.397065 8546 raft_consensus.cc:385] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.397105 8546 raft_consensus.cc:740] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.397248 8546 consensus_queue.cc:260] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.397408 8552 ts_tablet_manager.cc:1916] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.397734 8545 tablet_bootstrap.cc:654] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.397858 8546 ts_tablet_manager.cc:1434] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251028 02:59:53.398061 8546 tablet_bootstrap.cc:492] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.399096 8546 tablet_bootstrap.cc:654] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.399039 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2f140d5674bf4d588e435e79e52ec016" } new_table_name: "default.Table_83d3bc102e114c3eb1bb7e3c34291f8c" modify_external_catalogs: true
I20251028 02:59:53.399896 8552 ts_tablet_manager.cc:1929] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.399956 8552 log.cc:1199] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3057f4749edd43ac889f64d1f4e282d1
I20251028 02:59:53.400208 8552 ts_tablet_manager.cc:1950] T 3057f4749edd43ac889f64d1f4e282d1 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.401767 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3057f4749edd43ac889f64d1f4e282d1 (table default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]) successfully deleted
I20251028 02:59:53.404199 8550 ts_tablet_manager.cc:1929] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.404271 8550 log.cc:1199] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2bcbf3c72bb34c6291f4c1b1b9b1e906
I20251028 02:59:53.404472 8550 ts_tablet_manager.cc:1950] T 2bcbf3c72bb34c6291f4c1b1b9b1e906 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.404892 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2bcbf3c72bb34c6291f4c1b1b9b1e906 (table default.Table_3adef41531ff4ba0b4f92f0bd13ad3da [id=683ab7798036416caf92fc0102c53745]) successfully deleted
I20251028 02:59:53.405412 8544 tablet_bootstrap.cc:492] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.405496 8544 ts_tablet_manager.cc:1403] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20251028 02:59:53.406021 8544 raft_consensus.cc:359] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.406126 8544 raft_consensus.cc:385] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.406157 8544 raft_consensus.cc:740] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.406296 8544 consensus_queue.cc:260] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.406550 8544 ts_tablet_manager.cc:1434] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.406733 8544 tablet_bootstrap.cc:492] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.408103 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_322bdb2bdb474e39937bd5b0ecf1556e" } modify_external_catalogs: true
I20251028 02:59:53.408221 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_322bdb2bdb474e39937bd5b0ecf1556e" } modify_external_catalogs: true
W20251028 02:59:53.408977 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 617ca0f545824d14ba7270619f121bcc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 617ca0f545824d14ba7270619f121bcc already in progress: creating tablet
I20251028 02:59:53.411964 9023 tablet.cc:1722] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:53.412346 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet de38b2d54cec4f4e892047f4650f611b from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.412561 8544 tablet_bootstrap.cc:654] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.412956 9048 tablet.cc:1722] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:53.414151 8884 tablet.cc:1722] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:53.417624 8553 raft_consensus.cc:493] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.417718 8553 raft_consensus.cc:515] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.418092 8553 leader_election.cc:290] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.418408 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76f4055230904a25a1241a0a86a059d2" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:53.418644 8220 leader_election.cc:343] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 76f4055230904a25a1241a0a86a059d2
W20251028 02:59:53.418748 8218 leader_election.cc:343] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.418818 8218 leader_election.cc:304] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.418954 8553 raft_consensus.cc:2749] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.419723 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7ad84b67d9424145aa8f2efd29320e2f" } modify_external_catalogs: true
I20251028 02:59:53.419860 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7ad84b67d9424145aa8f2efd29320e2f" } modify_external_catalogs: true
I20251028 02:59:53.422308 8544 tablet_bootstrap.cc:492] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.422406 8544 ts_tablet_manager.cc:1403] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 02:59:53.422883 8544 raft_consensus.cc:359] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.422976 8544 raft_consensus.cc:385] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.423004 8544 raft_consensus.cc:740] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.423131 8544 consensus_queue.cc:260] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.423445 8544 ts_tablet_manager.cc:1434] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.423633 8544 tablet_bootstrap.cc:492] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.424680 8544 tablet_bootstrap.cc:654] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.426411 7965 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_8fdeffc6f6234305a829715139f150db [id=d4eaebd005bc42dd86d7125ef555546f] alter complete (version 2)
I20251028 02:59:53.426980 8122 tablet_service.cc:1505] Processing CreateTablet for tablet e71f5d12fe2e4822b14c72c2b880c623 (DEFAULT_TABLE table=default.Table_d0d760998d424df696eb71d3f8223366 [id=f1872634680f477e9f44007a30ef02bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.427253 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71f5d12fe2e4822b14c72c2b880c623. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.429014 8119 tablet_service.cc:1505] Processing CreateTablet for tablet a198db14385543d99afb1881b0fe2409 (DEFAULT_TABLE table=default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.429265 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a198db14385543d99afb1881b0fe2409. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.430262 8400 tablet_service.cc:1505] Processing CreateTablet for tablet e71f5d12fe2e4822b14c72c2b880c623 (DEFAULT_TABLE table=default.Table_d0d760998d424df696eb71d3f8223366 [id=f1872634680f477e9f44007a30ef02bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.430543 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71f5d12fe2e4822b14c72c2b880c623. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.432312 8397 tablet_service.cc:1505] Processing CreateTablet for tablet a198db14385543d99afb1881b0fe2409 (DEFAULT_TABLE table=default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.432564 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a198db14385543d99afb1881b0fe2409. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.433148 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet db6eadc5927d4876af9a50b44be97a63 with delete_type TABLET_DATA_DELETED (replaced by tablet 67c6ceefa3294af8b68150609f46c117) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.433460 8259 tablet_service.cc:1505] Processing CreateTablet for tablet e71f5d12fe2e4822b14c72c2b880c623 (DEFAULT_TABLE table=default.Table_d0d760998d424df696eb71d3f8223366 [id=f1872634680f477e9f44007a30ef02bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.433514 8261 tablet_service.cc:1505] Processing CreateTablet for tablet ac782a9f0b564667a7b20f1e328f2572 (DEFAULT_TABLE table=default.Table_114c71e193e84c098496acdd966b7fb7 [id=11719def996f48e68ac7d4ee2dbf66c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.433651 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 1ef353f5e715452eae1f1c191fe79d1d (DEFAULT_TABLE table=default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.433728 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71f5d12fe2e4822b14c72c2b880c623. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.433879 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef353f5e715452eae1f1c191fe79d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.434002 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac782a9f0b564667a7b20f1e328f2572. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.434706 8396 tablet_service.cc:1505] Processing CreateTablet for tablet ac782a9f0b564667a7b20f1e328f2572 (DEFAULT_TABLE table=default.Table_114c71e193e84c098496acdd966b7fb7 [id=11719def996f48e68ac7d4ee2dbf66c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.434968 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac782a9f0b564667a7b20f1e328f2572. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.436164 8260 tablet_service.cc:1505] Processing CreateTablet for tablet eaa41f8bf7c54673b65893acc59052c1 (DEFAULT_TABLE table=default.Table_33d362b6df3e40b68a3221d89a6b8358 [id=b4c56d84dddf453e96afc2c368def28e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.436378 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa41f8bf7c54673b65893acc59052c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.436618 8119 tablet_service.cc:1505] Processing CreateTablet for tablet ac782a9f0b564667a7b20f1e328f2572 (DEFAULT_TABLE table=default.Table_114c71e193e84c098496acdd966b7fb7 [id=11719def996f48e68ac7d4ee2dbf66c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.436856 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac782a9f0b564667a7b20f1e328f2572. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.437167 8258 tablet_service.cc:1505] Processing CreateTablet for tablet a198db14385543d99afb1881b0fe2409 (DEFAULT_TABLE table=default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.437399 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a198db14385543d99afb1881b0fe2409. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.438995 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 002de8aede994720b8163f41938af6c4 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.440239 8544 tablet_bootstrap.cc:492] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.440320 8544 ts_tablet_manager.cc:1403] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 02:59:53.440537 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 37dcf605a2984be1817d9621e4a41448 (table default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc]). Expected version 1 got 0
I20251028 02:59:53.440981 8544 raft_consensus.cc:359] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.441097 8544 raft_consensus.cc:385] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.441126 8544 raft_consensus.cc:740] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.441248 8544 consensus_queue.cc:260] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.441318 8122 tablet_service.cc:1505] Processing CreateTablet for tablet eaa41f8bf7c54673b65893acc59052c1 (DEFAULT_TABLE table=default.Table_33d362b6df3e40b68a3221d89a6b8358 [id=b4c56d84dddf453e96afc2c368def28e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.441560 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa41f8bf7c54673b65893acc59052c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.442260 8544 ts_tablet_manager.cc:1434] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:53.442454 8544 tablet_bootstrap.cc:492] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.443169 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 488bc52f1219478ab8843c9b1d79de2b
I20251028 02:59:53.443478 8544 tablet_bootstrap.cc:654] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.444027 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 488bc52f1219478ab8843c9b1d79de2b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.444027 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 488bc52f1219478ab8843c9b1d79de2b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.444200 8551 tablet_replica.cc:333] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.444324 8551 raft_consensus.cc:2243] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.444422 8551 raft_consensus.cc:2272] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.444775 8551 ts_tablet_manager.cc:1916] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.446710 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 488bc52f1219478ab8843c9b1d79de2b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.447149 8552 tablet_replica.cc:333] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.447324 8552 raft_consensus.cc:2243] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:53.447431 8220 consensus_peers.cc:597] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:53.447643 8400 tablet_service.cc:1505] Processing CreateTablet for tablet eaa41f8bf7c54673b65893acc59052c1 (DEFAULT_TABLE table=default.Table_33d362b6df3e40b68a3221d89a6b8358 [id=b4c56d84dddf453e96afc2c368def28e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.447927 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa41f8bf7c54673b65893acc59052c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.448119 8551 ts_tablet_manager.cc:1929] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.448184 8551 log.cc:1199] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/488bc52f1219478ab8843c9b1d79de2b
I20251028 02:59:53.448395 8551 ts_tablet_manager.cc:1950] T 488bc52f1219478ab8843c9b1d79de2b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.448866 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 309a9d10178e4984a18e10dfd11cba22 (DEFAULT_TABLE table=default.Table_c6ab20b423864891ba7a399af893a205 [id=4e185be67d91416e835dfc1c32e7e1e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.449018 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 488bc52f1219478ab8843c9b1d79de2b (table default.Table_7ad84b67d9424145aa8f2efd29320e2f [id=d8299d574efc4f3f9f40b93d5db5e430]) successfully deleted
I20251028 02:59:53.449121 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309a9d10178e4984a18e10dfd11cba22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.449271 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 1ef353f5e715452eae1f1c191fe79d1d (DEFAULT_TABLE table=default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.449499 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef353f5e715452eae1f1c191fe79d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.450071 8550 tablet_replica.cc:333] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.450196 8550 raft_consensus.cc:2243] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.450289 8550 raft_consensus.cc:2272] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.450755 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet a923402bbc734a0cb44aaf590d1e68ff from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.451350 8544 tablet_bootstrap.cc:492] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.451431 8544 ts_tablet_manager.cc:1403] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 02:59:53.451840 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 1ef353f5e715452eae1f1c191fe79d1d (DEFAULT_TABLE table=default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.451915 8544 raft_consensus.cc:359] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.452013 8544 raft_consensus.cc:385] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.452047 8544 raft_consensus.cc:740] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.452103 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef353f5e715452eae1f1c191fe79d1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.452167 8544 consensus_queue.cc:260] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.452562 8122 tablet_service.cc:1505] Processing CreateTablet for tablet a8f43f58ec624ca4bb12835f2e0eecbf (DEFAULT_TABLE table=default.Table_52becc2f534b4a0ab52b932cf5d2aec0 [id=2f8c8bd6c79d43c894e6356c6e333e7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.452793 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8f43f58ec624ca4bb12835f2e0eecbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.453971 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 309a9d10178e4984a18e10dfd11cba22 (DEFAULT_TABLE table=default.Table_c6ab20b423864891ba7a399af893a205 [id=4e185be67d91416e835dfc1c32e7e1e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.454192 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309a9d10178e4984a18e10dfd11cba22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.454931 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 309a9d10178e4984a18e10dfd11cba22 (DEFAULT_TABLE table=default.Table_c6ab20b423864891ba7a399af893a205 [id=4e185be67d91416e835dfc1c32e7e1e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.455185 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309a9d10178e4984a18e10dfd11cba22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.455770 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3ed9eb1b206e4d69ae9e4113517486f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.456157 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ed9eb1b206e4d69ae9e4113517486f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.456967 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet de38b2d54cec4f4e892047f4650f611b
I20251028 02:59:53.457008 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 4416e1e10f53401cbddc525d9556d107 (DEFAULT_TABLE table=default.Table_322bdb2bdb474e39937bd5b0ecf1556e [id=2b814ef3eb474d1fa873eaeff4fa1799]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.457265 7967 catalog_manager.cc:6719] ReplaceTablet: tablet de38b2d54cec4f4e892047f4650f611b deleted and replaced by tablet b93581f0eb7e499597c4e5d2f6c058aa
I20251028 02:59:53.457446 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet de38b2d54cec4f4e892047f4650f611b with delete_type TABLET_DATA_DELETED (replaced by tablet b93581f0eb7e499597c4e5d2f6c058aa) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.457640 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4416e1e10f53401cbddc525d9556d107. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.458417 8550 ts_tablet_manager.cc:1916] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.458989 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8fdeffc6f6234305a829715139f150db" } new_table_name: "default.Table_758c1adf3eb6460d9fa4e675a42507e3" modify_external_catalogs: true
I20251028 02:59:53.459028 8400 tablet_service.cc:1505] Processing CreateTablet for tablet a8f43f58ec624ca4bb12835f2e0eecbf (DEFAULT_TABLE table=default.Table_52becc2f534b4a0ab52b932cf5d2aec0 [id=2f8c8bd6c79d43c894e6356c6e333e7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.459267 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8f43f58ec624ca4bb12835f2e0eecbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.459920 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4416e1e10f53401cbddc525d9556d107
I20251028 02:59:53.460394 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 4416e1e10f53401cbddc525d9556d107 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.460600 8550 ts_tablet_manager.cc:1929] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.460667 8550 log.cc:1199] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/488bc52f1219478ab8843c9b1d79de2b
I20251028 02:59:53.460904 8550 ts_tablet_manager.cc:1950] T 488bc52f1219478ab8843c9b1d79de2b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.460918 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_df1c66f50ffa4cdf81fe0f8d721c2a2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.461280 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df1c66f50ffa4cdf81fe0f8d721c2a2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.461272 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet de38b2d54cec4f4e892047f4650f611b with delete_type TABLET_DATA_DELETED (replaced by tablet b93581f0eb7e499597c4e5d2f6c058aa) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.447450 8552 raft_consensus.cc:2272] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.462333 8552 ts_tablet_manager.cc:1916] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.462447 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 4416e1e10f53401cbddc525d9556d107 (DEFAULT_TABLE table=default.Table_322bdb2bdb474e39937bd5b0ecf1556e [id=2b814ef3eb474d1fa873eaeff4fa1799]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.462488 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 4416e1e10f53401cbddc525d9556d107 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.462872 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4416e1e10f53401cbddc525d9556d107. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.463531 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ec3580a0e3af454f9154d6733e07ca55" } new_table_name: "default.Table_6aba0ca2a3074aac8a0bf7f2da936e1a" modify_external_catalogs: true
I20251028 02:59:53.464967 8552 ts_tablet_manager.cc:1929] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.465029 8552 log.cc:1199] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/db6eadc5927d4876af9a50b44be97a63
I20251028 02:59:53.465251 8552 ts_tablet_manager.cc:1950] T db6eadc5927d4876af9a50b44be97a63 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 02:59:53.465744 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet de38b2d54cec4f4e892047f4650f611b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de38b2d54cec4f4e892047f4650f611b already in progress: creating tablet
W20251028 02:59:53.465865 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4416e1e10f53401cbddc525d9556d107 because the tablet was not found. No further retry: Not found: Tablet not found: 4416e1e10f53401cbddc525d9556d107
I20251028 02:59:53.465878 8552 tablet_replica.cc:333] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.465854 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 9feee73521df42ae89f5af5c5eacf9bd (DEFAULT_TABLE table=default.Table_8cae8460736e4dcb9108757c62255907 [id=8d264830878e487daa057d286f1e3e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.465999 8552 raft_consensus.cc:2243] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:53.466122 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9feee73521df42ae89f5af5c5eacf9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.466183 8552 pending_rounds.cc:70] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:53.466212 8552 pending_rounds.cc:77] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597129605419008 op_type: NO_OP noop_request { }
I20251028 02:59:53.466264 8552 raft_consensus.cc:2889] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:53.466315 8552 raft_consensus.cc:2272] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.466871 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet de38b2d54cec4f4e892047f4650f611b with delete_type TABLET_DATA_DELETED (replaced by tablet b93581f0eb7e499597c4e5d2f6c058aa) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.466995 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 4416e1e10f53401cbddc525d9556d107 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.467067 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 4416e1e10f53401cbddc525d9556d107 (DEFAULT_TABLE table=default.Table_322bdb2bdb474e39937bd5b0ecf1556e [id=2b814ef3eb474d1fa873eaeff4fa1799]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.467361 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4416e1e10f53401cbddc525d9556d107. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.467895 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet db6eadc5927d4876af9a50b44be97a63 (table default.Table_12396134528e4544a80934aa91d07353 [id=86c2f59df3dd4d2195d18835dceff79b]) successfully deleted
I20251028 02:59:53.468180 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a72c82e0690d484c952d3a1a78883bc5" } modify_external_catalogs: true
I20251028 02:59:53.468283 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a72c82e0690d484c952d3a1a78883bc5" } modify_external_catalogs: true
I20251028 02:59:53.468269 8260 tablet_service.cc:1505] Processing CreateTablet for tablet a8f43f58ec624ca4bb12835f2e0eecbf (DEFAULT_TABLE table=default.Table_52becc2f534b4a0ab52b932cf5d2aec0 [id=2f8c8bd6c79d43c894e6356c6e333e7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.468529 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8f43f58ec624ca4bb12835f2e0eecbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.469075 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 9feee73521df42ae89f5af5c5eacf9bd (DEFAULT_TABLE table=default.Table_8cae8460736e4dcb9108757c62255907 [id=8d264830878e487daa057d286f1e3e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.469274 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9feee73521df42ae89f5af5c5eacf9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.470458 8552 ts_tablet_manager.cc:1916] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.470470 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 8a81c8bc182442bd8596251b99f90a2d (DEFAULT_TABLE table=default.Table_63b74aebe1364823bb6a10f4a8942db8 [id=485d972ab1864e26ace68ba52bb671ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.470691 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a81c8bc182442bd8596251b99f90a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.472577 8552 ts_tablet_manager.cc:1929] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.472643 8552 log.cc:1199] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/488bc52f1219478ab8843c9b1d79de2b
I20251028 02:59:53.472929 8552 ts_tablet_manager.cc:1950] T 488bc52f1219478ab8843c9b1d79de2b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.472919 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 9feee73521df42ae89f5af5c5eacf9bd (DEFAULT_TABLE table=default.Table_8cae8460736e4dcb9108757c62255907 [id=8d264830878e487daa057d286f1e3e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.473186 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9feee73521df42ae89f5af5c5eacf9bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.473594 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 488bc52f1219478ab8843c9b1d79de2b (table default.Table_7ad84b67d9424145aa8f2efd29320e2f [id=d8299d574efc4f3f9f40b93d5db5e430]) successfully deleted
W20251028 02:59:53.474179 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet de38b2d54cec4f4e892047f4650f611b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de38b2d54cec4f4e892047f4650f611b already in progress: creating tablet
W20251028 02:59:53.474311 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4416e1e10f53401cbddc525d9556d107 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4416e1e10f53401cbddc525d9556d107 already in progress: creating tablet
I20251028 02:59:53.475032 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 8a81c8bc182442bd8596251b99f90a2d (DEFAULT_TABLE table=default.Table_63b74aebe1364823bb6a10f4a8942db8 [id=485d972ab1864e26ace68ba52bb671ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.475075 8122 tablet_service.cc:1505] Processing CreateTablet for tablet ce64e4407ad048e8add422d77de6c211 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.475401 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce64e4407ad048e8add422d77de6c211. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.476101 8656 raft_consensus.cc:493] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.476174 8656 raft_consensus.cc:515] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.476446 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 488bc52f1219478ab8843c9b1d79de2b (table default.Table_7ad84b67d9424145aa8f2efd29320e2f [id=d8299d574efc4f3f9f40b93d5db5e430]) successfully deleted
I20251028 02:59:53.476476 8656 leader_election.cc:290] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.476720 8544 ts_tablet_manager.cc:1434] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.025s user 0.001s sys 0.000s
I20251028 02:59:53.476881 8544 tablet_bootstrap.cc:492] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 02:59:53.477694 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet de38b2d54cec4f4e892047f4650f611b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de38b2d54cec4f4e892047f4650f611b already in progress: creating tablet
W20251028 02:59:53.477849 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4416e1e10f53401cbddc525d9556d107 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4416e1e10f53401cbddc525d9556d107 already in progress: creating tablet
I20251028 02:59:53.475288 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a81c8bc182442bd8596251b99f90a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.478812 8259 tablet_service.cc:1505] Processing CreateTablet for tablet ce64e4407ad048e8add422d77de6c211 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.479063 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce64e4407ad048e8add422d77de6c211. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.479074 8545 tablet_bootstrap.cc:492] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.479172 8545 ts_tablet_manager.cc:1403] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.082s user 0.002s sys 0.000s
I20251028 02:59:53.479696 8545 raft_consensus.cc:359] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.479805 8545 raft_consensus.cc:385] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.479842 8545 raft_consensus.cc:740] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.479983 8545 consensus_queue.cc:260] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.480233 8553 raft_consensus.cc:493] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.480278 8545 ts_tablet_manager.cc:1434] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:53.480315 8553 raft_consensus.cc:515] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.480502 8545 tablet_bootstrap.cc:492] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.480695 8553 leader_election.cc:290] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.481528 8545 tablet_bootstrap.cc:654] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.481801 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58934e83e36e42e7b913f01b64bd7e70" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.481935 8421 raft_consensus.cc:2468] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:53.482753 8220 leader_election.cc:304] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.483872 8549 raft_consensus.cc:493] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.483950 8549 raft_consensus.cc:515] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.484300 8549 leader_election.cc:290] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.484562 8553 raft_consensus.cc:2804] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.484613 8553 raft_consensus.cc:493] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.484649 8553 raft_consensus.cc:3060] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.485759 8553 raft_consensus.cc:515] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.486143 8553 leader_election.cc:290] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.486639 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58934e83e36e42e7b913f01b64bd7e70" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:53.486747 8421 raft_consensus.cc:3060] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.487768 8546 tablet_bootstrap.cc:492] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.487855 8546 ts_tablet_manager.cc:1403] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.090s user 0.002s sys 0.000s
I20251028 02:59:53.487964 8421 raft_consensus.cc:2468] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:53.488332 8546 raft_consensus.cc:359] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.488422 8546 raft_consensus.cc:385] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.488451 8546 raft_consensus.cc:740] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.488572 8546 consensus_queue.cc:260] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.488844 8546 ts_tablet_manager.cc:1434] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.489034 8546 tablet_bootstrap.cc:492] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.490037 8546 tablet_bootstrap.cc:654] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.490653 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7e347497892b4add9f7b4e19bde67878" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.490764 8220 leader_election.cc:304] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.490983 8556 raft_consensus.cc:2804] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:53.491063 8556 raft_consensus.cc:697] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:53.491029 8400 tablet_service.cc:1505] Processing CreateTablet for tablet ce64e4407ad048e8add422d77de6c211 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.491206 8556 consensus_queue.cc:237] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.491339 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce64e4407ad048e8add422d77de6c211. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.490782 8285 raft_consensus.cc:2468] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:53.491659 8356 leader_election.cc:304] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.491842 8549 raft_consensus.cc:2804] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.491891 8549 raft_consensus.cc:493] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.491928 8549 raft_consensus.cc:3060] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.492884 8549 raft_consensus.cc:515] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.493228 7965 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_83d3bc102e114c3eb1bb7e3c34291f8c [id=b0eb2947677348c9a4f7128df96afbbf] alter complete (version 1)
I20251028 02:59:53.493268 8549 leader_election.cc:290] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.493716 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a923402bbc734a0cb44aaf590d1e68ff
I20251028 02:59:53.493927 7962 catalog_manager.cc:6719] ReplaceTablet: tablet a923402bbc734a0cb44aaf590d1e68ff deleted and replaced by tablet 9e260030c0e6423cba37a2e3af1c6400
I20251028 02:59:53.494464 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7e347497892b4add9f7b4e19bde67878" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:53.494571 8285 raft_consensus.cc:3060] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.495496 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 76f4055230904a25a1241a0a86a059d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.495559 8285 raft_consensus.cc:2468] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:53.495661 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 50a0a3d690154de7acd55f37ab736d13 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ff8c974231f4b8aa18c18a05ad3e738) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.495976 8356 leader_election.cc:304] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.496145 8549 raft_consensus.cc:2804] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:53.496223 8549 raft_consensus.cc:697] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:53.496356 8549 consensus_queue.cc:237] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.496485 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f64a73f0705a4a119870d8d17497b338"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.496896 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f64a73f0705a4a119870d8d17497b338 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.497439 8545 tablet_bootstrap.cc:492] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.497526 8545 ts_tablet_manager.cc:1403] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20251028 02:59:53.497992 8545 raft_consensus.cc:359] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.498100 8545 raft_consensus.cc:385] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.498140 8545 raft_consensus.cc:740] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.498251 8545 consensus_queue.cc:260] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.498539 8545 ts_tablet_manager.cc:1434] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.498703 8545 tablet_bootstrap.cc:492] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.499497 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet a923402bbc734a0cb44aaf590d1e68ff with delete_type TABLET_DATA_DELETED (replaced by tablet 9e260030c0e6423cba37a2e3af1c6400) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.500854 8545 tablet_bootstrap.cc:654] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.503268 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet aec9427ed4284d66b9bd46c0e1c8774c with delete_type TABLET_DATA_DELETED (replaced by tablet de38b2d54cec4f4e892047f4650f611b) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.504776 8545 tablet_bootstrap.cc:492] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.504858 8545 ts_tablet_manager.cc:1403] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:53.505385 8545 raft_consensus.cc:359] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.505484 8545 raft_consensus.cc:385] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.505517 8545 raft_consensus.cc:740] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.505631 8545 consensus_queue.cc:260] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.505905 8545 ts_tablet_manager.cc:1434] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.506103 8545 tablet_bootstrap.cc:492] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.506796 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet b1b505f76ef84af6bc2b78a962bf6cf9
I20251028 02:59:53.507892 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet b1b505f76ef84af6bc2b78a962bf6cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.508591 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet b1b505f76ef84af6bc2b78a962bf6cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.508880 8545 tablet_bootstrap.cc:654] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:53.478015 8083 leader_election.cc:343] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: 2d711941006d4417901135b7761a251f
W20251028 02:59:53.509162 8083 leader_election.cc:343] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 2d711941006d4417901135b7761a251f
I20251028 02:59:53.509220 8083 leader_election.cc:304] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:53.509829 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 76f4055230904a25a1241a0a86a059d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.509927 8562 raft_consensus.cc:2749] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.509979 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet a923402bbc734a0cb44aaf590d1e68ff with delete_type TABLET_DATA_DELETED (replaced by tablet 9e260030c0e6423cba37a2e3af1c6400) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.510039 8550 tablet_replica.cc:333] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.510090 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet b1b505f76ef84af6bc2b78a962bf6cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.510341 8562 raft_consensus.cc:493] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.510422 8562 raft_consensus.cc:515] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.510766 8562 leader_election.cc:290] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.483791 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58934e83e36e42e7b913f01b64bd7e70" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.488608 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58934e83e36e42e7b913f01b64bd7e70" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:53.511461 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f57b4fa3ceec4557bc6d6a2125d80790" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.497174 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7e347497892b4add9f7b4e19bde67878" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:53.511610 8283 raft_consensus.cc:2393] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 02:59:53.499748 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7e347497892b4add9f7b4e19bde67878" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.506400 8656 raft_consensus.cc:493] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.511884 8656 raft_consensus.cc:515] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.512197 8656 leader_election.cc:290] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.477814 8544 tablet_bootstrap.cc:654] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.513630 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f57b4fa3ceec4557bc6d6a2125d80790" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.514004 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ce36e2a0a4374085b1e87288b3288ab9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.514242 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76f4055230904a25a1241a0a86a059d2" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.514353 8285 raft_consensus.cc:2468] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
W20251028 02:59:53.514436 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce36e2a0a4374085b1e87288b3288ab9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:53.514478 8083 leader_election.cc:343] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 76f4055230904a25a1241a0a86a059d2
I20251028 02:59:53.514617 8083 leader_election.cc:304] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:53.514864 8083 leader_election.cc:304] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:53.515051 8562 raft_consensus.cc:3060] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.515081 8656 raft_consensus.cc:2804] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.515141 8656 raft_consensus.cc:493] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.515100 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_fa500fb2e478468fac29b3a6682b2283"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.515178 8656 raft_consensus.cc:3060] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.515316 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ba1740929d404081b7451d23bc910475"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.515466 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa500fb2e478468fac29b3a6682b2283 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:53.515609 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba1740929d404081b7451d23bc910475 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.516680 8551 tablet_replica.cc:333] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.516826 8551 raft_consensus.cc:2243] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.516945 8551 raft_consensus.cc:2272] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.517228 8616 raft_consensus.cc:493] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.517305 8616 raft_consensus.cc:515] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.517572 8551 ts_tablet_manager.cc:1916] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.517633 8616 leader_election.cc:290] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.517904 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet e359f0121c334cc1a8b3eedbbe64bda2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.518136 8562 raft_consensus.cc:2749] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.518108 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aec9427ed4284d66b9bd46c0e1c8774c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.518421 8656 raft_consensus.cc:515] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:53.518515 8218 leader_election.cc:343] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.518628 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet a923402bbc734a0cb44aaf590d1e68ff with delete_type TABLET_DATA_DELETED (replaced by tablet 9e260030c0e6423cba37a2e3af1c6400) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.518481 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 8a81c8bc182442bd8596251b99f90a2d (DEFAULT_TABLE table=default.Table_63b74aebe1364823bb6a10f4a8942db8 [id=485d972ab1864e26ace68ba52bb671ad]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:53.518689 8218 leader_election.cc:343] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.518779 8656 leader_election.cc:290] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.518895 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a81c8bc182442bd8596251b99f90a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.519150 8550 raft_consensus.cc:2243] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.519495 8550 raft_consensus.cc:2272] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.519513 8884 tablet.cc:1722] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20251028 02:59:53.520064 8356 leader_election.cc:343] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.520641 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bcdd1e4fff1a45e39b03d0c7f2268672" } new_table_name: "default.Table_6d8f283a536e46fcbf3d8abcf9d703bb" modify_external_catalogs: true
I20251028 02:59:53.521119 8884 tablet.cc:1722] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:53.521361 9048 tablet.cc:1722] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:53.521483 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_52becc2f534b4a0ab52b932cf5d2aec0" } new_table_name: "default.Table_e2689b73a8454c8dbeab470093ad1463" modify_external_catalogs: true
I20251028 02:59:53.521456 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0d1a3e0b91aa41dcb3264e04169fdd5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.521821 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0d1a3e0b91aa41dcb3264e04169fdd5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.521900 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "76f4055230904a25a1241a0a86a059d2" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:53.522001 8285 raft_consensus.cc:3060] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.523021 8285 raft_consensus.cc:2468] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:53.523728 8550 ts_tablet_manager.cc:1916] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.524364 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aec9427ed4284d66b9bd46c0e1c8774c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.524909 9023 tablet.cc:1722] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.525975 8550 ts_tablet_manager.cc:1929] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.526041 8550 log.cc:1199] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/76f4055230904a25a1241a0a86a059d2
I20251028 02:59:53.526260 8550 ts_tablet_manager.cc:1950] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.526356 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_424dccd707fc4fa1bd9d01e78cab2f92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.526737 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_424dccd707fc4fa1bd9d01e78cab2f92 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.526770 8550 tablet_replica.cc:333] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.526929 8550 raft_consensus.cc:2243] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.527040 8550 raft_consensus.cc:2272] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.527287 8083 leader_election.cc:304] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
W20251028 02:59:53.527419 8083 raft_consensus.cc:2686] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251028 02:59:53.527539 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 76f4055230904a25a1241a0a86a059d2 (table default.Table_d2e5ac67a4414c03b7890dea514caeee [id=c8f1a159643642eb8dc2f202d73532fe]) successfully deleted
W20251028 02:59:53.527693 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a923402bbc734a0cb44aaf590d1e68ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a923402bbc734a0cb44aaf590d1e68ff already in progress: creating tablet
I20251028 02:59:53.527828 8616 raft_consensus.cc:493] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.527935 8550 ts_tablet_manager.cc:1916] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.527915 8616 raft_consensus.cc:515] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.528264 8616 leader_election.cc:290] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.528648 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da1563d65b4d4ab48325a4f08ee89ed5" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:53.521402 8354 leader_election.cc:343] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:53.529162 8354 leader_election.cc:343] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:53.528963 8356 leader_election.cc:343] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:53.529983 8083 leader_election.cc:343] T 76f4055230904a25a1241a0a86a059d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 76f4055230904a25a1241a0a86a059d2
I20251028 02:59:53.530148 8550 ts_tablet_manager.cc:1929] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.530223 8550 log.cc:1199] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/b1b505f76ef84af6bc2b78a962bf6cf9
I20251028 02:59:53.530490 8550 ts_tablet_manager.cc:1950] T b1b505f76ef84af6bc2b78a962bf6cf9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.530639 8551 ts_tablet_manager.cc:1929] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.530710 8551 log.cc:1199] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/50a0a3d690154de7acd55f37ab736d13
I20251028 02:59:53.530939 8551 ts_tablet_manager.cc:1950] T 50a0a3d690154de7acd55f37ab736d13 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.531033 8550 tablet_replica.cc:333] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.531158 8550 raft_consensus.cc:2243] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.531245 8550 raft_consensus.cc:2272] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.531591 8551 tablet_replica.cc:333] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.531720 8551 raft_consensus.cc:2243] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.531821 8551 raft_consensus.cc:2272] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.531965 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 50a0a3d690154de7acd55f37ab736d13 (table default.Table_b9407dac1631402082afb2b12dc61dbd [id=a015c31e18c74191879fe1474dcb6ab2]) successfully deleted
I20251028 02:59:53.532243 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet b1b505f76ef84af6bc2b78a962bf6cf9 (table default.Table_a72c82e0690d484c952d3a1a78883bc5 [id=32b09554787b45c0811966bfb3f70031]) successfully deleted
I20251028 02:59:53.532303 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "da1563d65b4d4ab48325a4f08ee89ed5" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.532500 9023 tablet.cc:1722] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:53.533821 8556 raft_consensus.cc:493] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.533908 8556 raft_consensus.cc:515] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.534307 8556 leader_election.cc:290] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:53.535250 8220 leader_election.cc:343] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 374936c130c841348e0d76cce070a295
I20251028 02:59:53.535758 8550 ts_tablet_manager.cc:1916] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.537195 8656 raft_consensus.cc:493] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.537278 8656 raft_consensus.cc:515] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.537616 8656 leader_election.cc:290] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.537822 9048 tablet.cc:1722] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.537969 8550 ts_tablet_manager.cc:1929] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.538033 8550 log.cc:1199] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/e359f0121c334cc1a8b3eedbbe64bda2
I20251028 02:59:53.538249 8550 ts_tablet_manager.cc:1950] T e359f0121c334cc1a8b3eedbbe64bda2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 02:59:53.538283 8218 leader_election.cc:343] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 374936c130c841348e0d76cce070a295
I20251028 02:59:53.538357 8218 leader_election.cc:304] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.538434 8551 ts_tablet_manager.cc:1916] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.539093 8616 raft_consensus.cc:493] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.539181 8616 raft_consensus.cc:515] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.539535 8616 leader_election.cc:290] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 02:59:53.539100 8354 leader_election.cc:343] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.539923 8354 leader_election.cc:304] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:53.540132 8354 leader_election.cc:343] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.540182 8354 leader_election.cc:304] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.540839 8616 raft_consensus.cc:2749] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.540953 8616 raft_consensus.cc:2749] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.541298 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37dcf605a2984be1817d9621e4a41448" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.541430 8553 raft_consensus.cc:2749] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.541549 8553 raft_consensus.cc:493] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.541610 8553 raft_consensus.cc:515] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.541975 8553 leader_election.cc:290] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.542428 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c1d8552cc1941bda2e9f02fba56c9f9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.542474 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c1d8552cc1941bda2e9f02fba56c9f9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.542516 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37dcf605a2984be1817d9621e4a41448" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.542541 8421 raft_consensus.cc:2468] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:53.542564 8145 raft_consensus.cc:2468] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
W20251028 02:59:53.542816 8356 leader_election.cc:343] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.542817 8220 leader_election.cc:304] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.542972 8556 raft_consensus.cc:2804] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.543020 8556 raft_consensus.cc:493] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.543058 8556 raft_consensus.cc:3060] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:53.543253 8354 leader_election.cc:343] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.543329 8354 leader_election.cc:304] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.544181 8556 raft_consensus.cc:515] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.544755 8556 leader_election.cc:290] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.545240 8549 raft_consensus.cc:2749] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.545264 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c1d8552cc1941bda2e9f02fba56c9f9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:53.545360 8145 raft_consensus.cc:3060] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.545430 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c1d8552cc1941bda2e9f02fba56c9f9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:53.545526 8421 raft_consensus.cc:3060] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.545881 8551 ts_tablet_manager.cc:1929] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.545943 8551 log.cc:1199] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a923402bbc734a0cb44aaf590d1e68ff
I20251028 02:59:53.546154 8551 ts_tablet_manager.cc:1950] T a923402bbc734a0cb44aaf590d1e68ff P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.546341 8145 raft_consensus.cc:2468] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:53.546478 8421 raft_consensus.cc:2468] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:53.546906 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a923402bbc734a0cb44aaf590d1e68ff (table default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]) successfully deleted
I20251028 02:59:53.547379 8220 leader_election.cc:304] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.547868 8551 tablet_replica.cc:333] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.548148 8551 raft_consensus.cc:2243] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.548144 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet e359f0121c334cc1a8b3eedbbe64bda2 (table default.Table_726e0c2d9d4044d8b79bb266e21a8d9c [id=6630b83265864f36ad74cac692f131e3]) successfully deleted
I20251028 02:59:53.548254 8551 raft_consensus.cc:2272] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.548290 8546 tablet_bootstrap.cc:492] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.548365 8546 ts_tablet_manager.cc:1403] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.059s user 0.001s sys 0.000s
I20251028 02:59:53.548545 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c1d8552cc1941bda2e9f02fba56c9f9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.548707 8285 raft_consensus.cc:2393] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:53.548817 8546 raft_consensus.cc:359] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.548904 8546 raft_consensus.cc:385] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.548931 8546 raft_consensus.cc:740] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.548979 8965 raft_consensus.cc:2804] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:53.549036 8546 consensus_queue.cc:260] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.549062 8965 raft_consensus.cc:697] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:53.549198 8965 consensus_queue.cc:237] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.549295 8546 ts_tablet_manager.cc:1434] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.549376 8551 ts_tablet_manager.cc:1916] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.549486 8546 tablet_bootstrap.cc:492] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.550626 8546 tablet_bootstrap.cc:654] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.551791 8551 ts_tablet_manager.cc:1929] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.551867 8551 log.cc:1199] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/aec9427ed4284d66b9bd46c0e1c8774c
I20251028 02:59:53.552062 8551 ts_tablet_manager.cc:1950] T aec9427ed4284d66b9bd46c0e1c8774c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.552657 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet aec9427ed4284d66b9bd46c0e1c8774c (table default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]) successfully deleted
I20251028 02:59:53.553261 8551 tablet_replica.cc:333] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.553386 8546 tablet_bootstrap.cc:492] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.553391 8551 raft_consensus.cc:2243] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.553462 8546 ts_tablet_manager.cc:1403] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20251028 02:59:53.553503 8551 raft_consensus.cc:2272] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.553887 8546 raft_consensus.cc:359] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.553982 8546 raft_consensus.cc:385] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.554013 8546 raft_consensus.cc:740] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.554097 8551 ts_tablet_manager.cc:1916] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.554119 8546 consensus_queue.cc:260] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.554375 8546 ts_tablet_manager.cc:1434] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.554559 8546 tablet_bootstrap.cc:492] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.554724 8553 consensus_queue.cc:1048] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.555536 8546 tablet_bootstrap.cc:654] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.556881 8551 ts_tablet_manager.cc:1929] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.556950 8551 log.cc:1199] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/b1b505f76ef84af6bc2b78a962bf6cf9
I20251028 02:59:53.557224 8551 ts_tablet_manager.cc:1950] T b1b505f76ef84af6bc2b78a962bf6cf9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.558894 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9c1d8552cc1941bda2e9f02fba56c9f9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.559031 8421 raft_consensus.cc:2393] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:53.559394 8083 leader_election.cc:304] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:53.559645 8656 raft_consensus.cc:2749] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.560261 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet b1b505f76ef84af6bc2b78a962bf6cf9 (table default.Table_a72c82e0690d484c952d3a1a78883bc5 [id=32b09554787b45c0811966bfb3f70031]) successfully deleted
I20251028 02:59:53.562131 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_aa25a85ac8d24940a727a586d6c630db"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.562595 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa25a85ac8d24940a727a586d6c630db in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.565198 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_df1c66f50ffa4cdf81fe0f8d721c2a2b" } modify_external_catalogs: true
I20251028 02:59:53.565322 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_df1c66f50ffa4cdf81fe0f8d721c2a2b" } modify_external_catalogs: true
I20251028 02:59:53.566812 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f2fce0c2e0b348e18d259ef0ee58e114"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.567173 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2fce0c2e0b348e18d259ef0ee58e114 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.573014 8552 tablet_replica.cc:333] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.573217 8552 raft_consensus.cc:2243] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.573343 8552 raft_consensus.cc:2272] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.573930 8552 ts_tablet_manager.cc:1916] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:53.574235 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a3096fde2ac94ab197ce5e659c4d2be0 consensus state has no leader
I20251028 02:59:53.574476 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_83d3bc102e114c3eb1bb7e3c34291f8c" } modify_external_catalogs: true
I20251028 02:59:53.574570 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_83d3bc102e114c3eb1bb7e3c34291f8c" } modify_external_catalogs: true
I20251028 02:59:53.575033 7964 catalog_manager.cc:5931] Not sending DeleteTablet requests; no consensus state for tablet c43be649c5c9419cab04800ac5465936
W20251028 02:59:53.575340 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a8f43f58ec624ca4bb12835f2e0eecbf consensus state has no leader
I20251028 02:59:53.580349 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 2b5908fd54944004866ca3864e8131d2 (DEFAULT_TABLE table=default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.580462 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 73c20437695a4043a86faa9c98091e5e (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.580641 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b5908fd54944004866ca3864e8131d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.580621 8400 tablet_service.cc:1505] Processing CreateTablet for tablet b93581f0eb7e499597c4e5d2f6c058aa (DEFAULT_TABLE table=default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.580818 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b93581f0eb7e499597c4e5d2f6c058aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.581058 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 2b5908fd54944004866ca3864e8131d2 (DEFAULT_TABLE table=default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.581229 8260 tablet_service.cc:1505] Processing CreateTablet for tablet b93581f0eb7e499597c4e5d2f6c058aa (DEFAULT_TABLE table=default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.581301 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b5908fd54944004866ca3864e8131d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.581413 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b93581f0eb7e499597c4e5d2f6c058aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.583348 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 73c20437695a4043a86faa9c98091e5e (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.583560 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73c20437695a4043a86faa9c98091e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.584403 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet d9c54e91d2d94573b40b50752318536d from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.585374 8552 ts_tablet_manager.cc:1929] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.585548 8552 log.cc:1199] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/76f4055230904a25a1241a0a86a059d2
I20251028 02:59:53.585773 8552 ts_tablet_manager.cc:1950] T 76f4055230904a25a1241a0a86a059d2 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.585863 8544 tablet_bootstrap.cc:492] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.585955 8544 ts_tablet_manager.cc:1403] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.109s user 0.002s sys 0.000s
I20251028 02:59:53.586426 8544 raft_consensus.cc:359] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.586526 8544 raft_consensus.cc:385] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.586552 8544 raft_consensus.cc:740] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.586602 8552 tablet_replica.cc:333] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.586660 8544 consensus_queue.cc:260] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.586728 8552 raft_consensus.cc:2243] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:53.586908 8544 ts_tablet_manager.cc:1434] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.586905 8552 raft_consensus.cc:2272] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.587098 8544 tablet_bootstrap.cc:492] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.587409 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73c20437695a4043a86faa9c98091e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.588058 8552 ts_tablet_manager.cc:1916] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.588362 8544 tablet_bootstrap.cc:654] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.589195 8546 tablet_bootstrap.cc:492] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.589278 8546 ts_tablet_manager.cc:1403] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.000s
I20251028 02:59:53.589705 8562 raft_consensus.cc:493] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.589699 8546 raft_consensus.cc:359] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.589823 8546 raft_consensus.cc:385] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.589852 8546 raft_consensus.cc:740] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.589840 8562 raft_consensus.cc:515] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.589962 8546 consensus_queue.cc:260] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.590197 8546 ts_tablet_manager.cc:1434] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.590226 8562 leader_election.cc:290] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.590348 8546 tablet_bootstrap.cc:492] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.591249 8546 tablet_bootstrap.cc:654] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.591523 8545 tablet_bootstrap.cc:492] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.591607 8545 ts_tablet_manager.cc:1403] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.086s user 0.002s sys 0.000s
I20251028 02:59:53.592106 8545 raft_consensus.cc:359] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.592217 8545 raft_consensus.cc:385] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.592254 8545 raft_consensus.cc:740] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.592368 8545 consensus_queue.cc:260] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.592440 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 76f4055230904a25a1241a0a86a059d2 (table default.Table_d2e5ac67a4414c03b7890dea514caeee [id=c8f1a159643642eb8dc2f202d73532fe]) successfully deleted
I20251028 02:59:53.592661 8545 ts_tablet_manager.cc:1434] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.592841 8145 raft_consensus.cc:3060] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.592862 8545 tablet_bootstrap.cc:492] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.593860 8545 tablet_bootstrap.cc:654] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.594187 8122 tablet_service.cc:1505] Processing CreateTablet for tablet b93581f0eb7e499597c4e5d2f6c058aa (DEFAULT_TABLE table=default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.594421 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b93581f0eb7e499597c4e5d2f6c058aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.594560 8544 tablet_bootstrap.cc:492] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.594626 8544 ts_tablet_manager.cc:1403] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:53.595050 8544 raft_consensus.cc:359] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.595142 8544 raft_consensus.cc:385] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.595067 9011 consensus_queue.cc:1048] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.595171 8544 raft_consensus.cc:740] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.595387 8544 consensus_queue.cc:260] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.596369 8544 ts_tablet_manager.cc:1434] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:53.596529 8544 tablet_bootstrap.cc:492] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.597209 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 73c20437695a4043a86faa9c98091e5e (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.597306 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet d107a510c8664999901320581e885ba9 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.597432 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73c20437695a4043a86faa9c98091e5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.597651 8544 tablet_bootstrap.cc:654] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.599383 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 2b5908fd54944004866ca3864e8131d2 (DEFAULT_TABLE table=default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.599604 8546 tablet_bootstrap.cc:492] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.599619 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b5908fd54944004866ca3864e8131d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.599692 8546 ts_tablet_manager.cc:1403] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 02:59:53.600924 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d9c54e91d2d94573b40b50752318536d
I20251028 02:59:53.600944 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4726573f4b994b2591332c4a42fb10a1
I20251028 02:59:53.601599 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 4726573f4b994b2591332c4a42fb10a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.601686 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9b92443ae55f4a489d92d6175b03f377" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.601907 7960 catalog_manager.cc:6719] ReplaceTablet: tablet d9c54e91d2d94573b40b50752318536d deleted and replaced by tablet 35e211a1a8c547baaee1199f7103bfad
I20251028 02:59:53.601948 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet d9c54e91d2d94573b40b50752318536d with delete_type TABLET_DATA_DELETED (replaced by tablet 35e211a1a8c547baaee1199f7103bfad) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.602308 8546 raft_consensus.cc:359] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.602401 8546 raft_consensus.cc:385] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.602428 8546 raft_consensus.cc:740] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.602531 8546 consensus_queue.cc:260] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.602768 8546 ts_tablet_manager.cc:1434] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:53.602949 8546 tablet_bootstrap.cc:492] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.603353 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet d9c54e91d2d94573b40b50752318536d with delete_type TABLET_DATA_DELETED (replaced by tablet 35e211a1a8c547baaee1199f7103bfad) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.604120 8546 tablet_bootstrap.cc:654] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.604677 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d107a510c8664999901320581e885ba9
I20251028 02:59:53.605000 7969 catalog_manager.cc:6719] ReplaceTablet: tablet d107a510c8664999901320581e885ba9 deleted and replaced by tablet 2a2ebe28dbd2439e963319ec6e515635
I20251028 02:59:53.605123 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet d107a510c8664999901320581e885ba9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a2ebe28dbd2439e963319ec6e515635) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:53.605494 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet d107a510c8664999901320581e885ba9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d107a510c8664999901320581e885ba9 already in progress: creating tablet
I20251028 02:59:53.605535 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 4726573f4b994b2591332c4a42fb10a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.605659 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet d9c54e91d2d94573b40b50752318536d with delete_type TABLET_DATA_DELETED (replaced by tablet 35e211a1a8c547baaee1199f7103bfad) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.605811 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9b92443ae55f4a489d92d6175b03f377" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.605854 8550 tablet_replica.cc:333] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.605988 8550 raft_consensus.cc:2243] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:53.606187 8550 raft_consensus.cc:2272] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.606429 8083 leader_election.cc:304] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:53.606693 8550 ts_tablet_manager.cc:1916] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.606791 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet d107a510c8664999901320581e885ba9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a2ebe28dbd2439e963319ec6e515635) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.607362 8656 raft_consensus.cc:2749] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.607628 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet d107a510c8664999901320581e885ba9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a2ebe28dbd2439e963319ec6e515635) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:53.608420 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet d9c54e91d2d94573b40b50752318536d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9c54e91d2d94573b40b50752318536d already in progress: creating tablet
I20251028 02:59:53.609195 8550 ts_tablet_manager.cc:1929] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:53.609253 8550 log.cc:1199] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4726573f4b994b2591332c4a42fb10a1
I20251028 02:59:53.609474 8550 ts_tablet_manager.cc:1950] T 4726573f4b994b2591332c4a42fb10a1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.610251 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4726573f4b994b2591332c4a42fb10a1 (table default.Table_83d3bc102e114c3eb1bb7e3c34291f8c [id=b0eb2947677348c9a4f7128df96afbbf]) successfully deleted
W20251028 02:59:53.610402 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet d9c54e91d2d94573b40b50752318536d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9c54e91d2d94573b40b50752318536d already in progress: creating tablet
W20251028 02:59:53.608578 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet d107a510c8664999901320581e885ba9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d107a510c8664999901320581e885ba9 already in progress: creating tablet
I20251028 02:59:53.611806 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e4f7aecd654a4a83b4435942625ca98a" } new_table_name: "default.Table_698073e4869e4f8fbbb8e8f8c548887d" modify_external_catalogs: true
I20251028 02:59:53.613229 8552 ts_tablet_manager.cc:1929] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.613305 8552 log.cc:1199] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/b1b505f76ef84af6bc2b78a962bf6cf9
I20251028 02:59:53.613575 8552 ts_tablet_manager.cc:1950] T b1b505f76ef84af6bc2b78a962bf6cf9 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.613590 8146 raft_consensus.cc:3060] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.614388 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet b1b505f76ef84af6bc2b78a962bf6cf9 (table default.Table_a72c82e0690d484c952d3a1a78883bc5 [id=32b09554787b45c0811966bfb3f70031]) successfully deleted
I20251028 02:59:53.615581 8552 tablet_replica.cc:333] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 02:59:53.615720 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a923402bbc734a0cb44aaf590d1e68ff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a923402bbc734a0cb44aaf590d1e68ff already in progress: creating tablet
I20251028 02:59:53.615881 8552 raft_consensus.cc:2243] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.615825 8549 consensus_queue.cc:1048] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.616071 8552 raft_consensus.cc:2272] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.616391 8545 tablet_bootstrap.cc:492] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.616501 8545 ts_tablet_manager.cc:1403] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20251028 02:59:53.616737 8552 ts_tablet_manager.cc:1916] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.617048 8545 raft_consensus.cc:359] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.617163 8545 raft_consensus.cc:385] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.617201 8545 raft_consensus.cc:740] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.617326 8545 consensus_queue.cc:260] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.617648 8545 ts_tablet_manager.cc:1434] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.617866 8545 tablet_bootstrap.cc:492] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.619371 8552 ts_tablet_manager.cc:1929] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:53.619562 8552 log.cc:1199] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4726573f4b994b2591332c4a42fb10a1
I20251028 02:59:53.619876 8552 ts_tablet_manager.cc:1950] T 4726573f4b994b2591332c4a42fb10a1 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.622586 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4726573f4b994b2591332c4a42fb10a1 (table default.Table_83d3bc102e114c3eb1bb7e3c34291f8c [id=b0eb2947677348c9a4f7128df96afbbf]) successfully deleted
W20251028 02:59:53.622762 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet d9c54e91d2d94573b40b50752318536d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9c54e91d2d94573b40b50752318536d already in progress: creating tablet
W20251028 02:59:53.622888 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet d107a510c8664999901320581e885ba9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d107a510c8664999901320581e885ba9 already in progress: creating tablet
I20251028 02:59:53.624760 7968 catalog_manager.cc:5649] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:53.625257 8545 tablet_bootstrap.cc:654] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.625394 8544 tablet_bootstrap.cc:492] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.625475 8544 ts_tablet_manager.cc:1403] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 02:59:53.625943 8544 raft_consensus.cc:359] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.626037 8544 raft_consensus.cc:385] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.626066 8544 raft_consensus.cc:740] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.626170 8544 consensus_queue.cc:260] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.626433 8544 ts_tablet_manager.cc:1434] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.626607 8544 tablet_bootstrap.cc:492] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.627526 8544 tablet_bootstrap.cc:654] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.628319 8545 tablet_bootstrap.cc:492] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.628409 8545 ts_tablet_manager.cc:1403] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:53.628765 9048 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.628890 8545 raft_consensus.cc:359] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.628978 8545 raft_consensus.cc:385] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.629006 8545 raft_consensus.cc:740] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.629134 8545 consensus_queue.cc:260] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.629432 8545 ts_tablet_manager.cc:1434] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.629616 8545 tablet_bootstrap.cc:492] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.630702 8545 tablet_bootstrap.cc:654] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.633952 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 02f6a32db68e4c609da39c3ee6c97348 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.634117 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 68ca4ff164b24b7498f953ec11cf46be (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:53.634171 8551 tablet_replica.cc:333] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.634311 8551 raft_consensus.cc:2243] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.634359 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 03959a61996b40d8b9ef8b4b9708f017 (table default.Table_5da94c10f01846a59e4ce4ff530330d3 [id=bc83d2963bb74c2d9b914dee53bcf6ba]). Expected version 1 got 0
I20251028 02:59:53.634425 8551 raft_consensus.cc:2272] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.634925 9122 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.636703 8551 ts_tablet_manager.cc:1916] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.637233 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 9c1d8552cc1941bda2e9f02fba56c9f9 (table default.Table_bb551b322c5140ba89d154ff450085dd [id=4e8c5072da4e4ec6a616ba4c7fc91cd3]). Expected version 1 got 0
I20251028 02:59:53.638358 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]). Expected version 1 got 0
I20251028 02:59:53.638484 7965 catalog_manager.cc:5649] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:53.639194 8551 ts_tablet_manager.cc:1929] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.639766 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 2d711941006d4417901135b7761a251f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.640015 8550 tablet_replica.cc:333] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.640148 8550 raft_consensus.cc:2243] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.640174 8117 tablet_service.cc:1505] Processing CreateTablet for tablet bb0e20b3e05c484394e11bbdea6a44ea (DEFAULT_TABLE table=default.Table_424dccd707fc4fa1bd9d01e78cab2f92 [id=dbe3cee1ce8a47fab35d8303c2950b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.640264 8550 raft_consensus.cc:2272] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.640478 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0e20b3e05c484394e11bbdea6a44ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.640805 8550 ts_tablet_manager.cc:1916] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.642441 8260 tablet_service.cc:1505] Processing CreateTablet for tablet bb0e20b3e05c484394e11bbdea6a44ea (DEFAULT_TABLE table=default.Table_424dccd707fc4fa1bd9d01e78cab2f92 [id=dbe3cee1ce8a47fab35d8303c2950b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.642736 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0e20b3e05c484394e11bbdea6a44ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.642812 8550 ts_tablet_manager.cc:1929] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.642867 8550 log.cc:1199] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2d711941006d4417901135b7761a251f
I20251028 02:59:53.643077 8550 ts_tablet_manager.cc:1950] T 2d711941006d4417901135b7761a251f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.644158 8398 tablet_service.cc:1505] Processing CreateTablet for tablet bb0e20b3e05c484394e11bbdea6a44ea (DEFAULT_TABLE table=default.Table_424dccd707fc4fa1bd9d01e78cab2f92 [id=dbe3cee1ce8a47fab35d8303c2950b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.644403 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb0e20b3e05c484394e11bbdea6a44ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.645298 8117 tablet_service.cc:1505] Processing CreateTablet for tablet df60914093f444fda88c13a132587513 (DEFAULT_TABLE table=default.Table_0d1a3e0b91aa41dcb3264e04169fdd5e [id=45e7b7c657e54d43a490443857404c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.645529 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df60914093f444fda88c13a132587513. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.647454 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c6ab20b423864891ba7a399af893a205" } modify_external_catalogs: true
I20251028 02:59:53.647567 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c6ab20b423864891ba7a399af893a205" } modify_external_catalogs: true
I20251028 02:59:53.648586 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2d711941006d4417901135b7761a251f (table default.Table_b4b24abe267c451f95fc63bf0234d06c [id=2e12a8e71ae3439badff5e4672007258]) successfully deleted
I20251028 02:59:53.649309 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_758c1adf3eb6460d9fa4e675a42507e3 [id=d4eaebd005bc42dd86d7125ef555546f] alter complete (version 3)
I20251028 02:59:53.639273 8551 log.cc:1199] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/02f6a32db68e4c609da39c3ee6c97348
I20251028 02:59:53.653802 8551 ts_tablet_manager.cc:1950] T 02f6a32db68e4c609da39c3ee6c97348 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.653765 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet da1563d65b4d4ab48325a4f08ee89ed5 (table default.Table_153d49d056a3458a9565c70d954187ae [id=ae8a178b437f4df4af15bb82bd7a95ee]). Expected version 1 got 0
I20251028 02:59:53.654022 7965 catalog_manager.cc:5649] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:53.654210 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 9c1d8552cc1941bda2e9f02fba56c9f9 (table default.Table_bb551b322c5140ba89d154ff450085dd [id=4e8c5072da4e4ec6a616ba4c7fc91cd3]). Expected version 1 got 0
I20251028 02:59:53.654327 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 37dcf605a2984be1817d9621e4a41448 (table default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc]). Expected version 1 got 0
I20251028 02:59:53.654587 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 02f6a32db68e4c609da39c3ee6c97348 (table default.Table_e69b7cea080549b1b8a67cb91703a472 [id=0c94092ef3844117af5e2aa5086dabb6]) successfully deleted
I20251028 02:59:53.654897 8545 tablet_bootstrap.cc:492] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.655184 8545 ts_tablet_manager.cc:1403] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20251028 02:59:53.655741 7968 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_698073e4869e4f8fbbb8e8f8c548887d [id=4eb2743632814baab2e39bbdce700d6b] alter complete (version 3)
I20251028 02:59:53.655928 8616 consensus_queue.cc:1048] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.656966 9011 raft_consensus.cc:493] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.657068 9011 raft_consensus.cc:515] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.657413 9011 leader_election.cc:290] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.658602 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50a0a3d690154de7acd55f37ab736d13" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:53.659765 8218 leader_election.cc:343] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.661180 9023 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.665452 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 309a9d10178e4984a18e10dfd11cba22
I20251028 02:59:53.666033 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_6aba0ca2a3074aac8a0bf7f2da936e1a [id=3eb0b70129be40e598874d492735ba51] alter complete (version 1)
I20251028 02:59:53.666620 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 309a9d10178e4984a18e10dfd11cba22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.667100 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 552c509b32374f5f9e3c232fbeddf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.667227 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 8a81c8bc182442bd8596251b99f90a2d from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:53.667542 8220 leader_election.cc:343] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 50a0a3d690154de7acd55f37ab736d13
I20251028 02:59:53.667622 8220 leader_election.cc:304] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.667845 9011 raft_consensus.cc:2749] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.668527 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet aeddf70e5bd342f19f3f427d1df73814 with delete_type TABLET_DATA_DELETED (replaced by tablet 5fd9cc1847ff4df7a09e2e698786d515) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.668599 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 50a0a3d690154de7acd55f37ab736d13 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ff8c974231f4b8aa18c18a05ad3e738) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.668726 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 309a9d10178e4984a18e10dfd11cba22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.668908 8550 tablet_replica.cc:333] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.669052 8550 raft_consensus.cc:2243] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.669158 8550 raft_consensus.cc:2272] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:53.669206 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 309a9d10178e4984a18e10dfd11cba22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 309a9d10178e4984a18e10dfd11cba22 already in progress: creating tablet
I20251028 02:59:53.669981 8545 raft_consensus.cc:359] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.670147 8545 raft_consensus.cc:385] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.670195 8545 raft_consensus.cc:740] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.670433 8545 consensus_queue.cc:260] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.670768 8545 ts_tablet_manager.cc:1434] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.016s user 0.001s sys 0.000s
W20251028 02:59:53.670835 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 309a9d10178e4984a18e10dfd11cba22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 309a9d10178e4984a18e10dfd11cba22 already in progress: creating tablet
I20251028 02:59:53.671020 9011 raft_consensus.cc:493] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.671123 9011 raft_consensus.cc:515] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.671478 8550 ts_tablet_manager.cc:1916] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.671533 9011 leader_election.cc:290] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.671752 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_153da548190a42a78cad7ae8553d6fba" } modify_external_catalogs: true
I20251028 02:59:53.671881 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_153da548190a42a78cad7ae8553d6fba" } modify_external_catalogs: true
W20251028 02:59:53.672266 8220 leader_election.cc:343] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: aeddf70e5bd342f19f3f427d1df73814
I20251028 02:59:53.672403 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 309a9d10178e4984a18e10dfd11cba22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:53.674270 7968 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet 68ca4ff164b24b7498f953ec11cf46be (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 68ca4ff164b24b7498f953ec11cf46be consensus state has no leader
I20251028 02:59:53.671025 8552 tablet_replica.cc:333] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.674878 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c87d03fad194ca89f1808da6dbdf0a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a3096fde2ac94ab197ce5e659c4d2be0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.675263 8552 raft_consensus.cc:2243] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.675547 8552 raft_consensus.cc:2272] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.676128 8550 ts_tablet_manager.cc:1929] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.676295 8550 log.cc:1199] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/552c509b32374f5f9e3c232fbeddf9bd
I20251028 02:59:53.676656 8550 ts_tablet_manager.cc:1950] T 552c509b32374f5f9e3c232fbeddf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.677626 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 552c509b32374f5f9e3c232fbeddf9bd (table default.Table_e3e2db3778b049ce8d0327aa81f7e2ae [id=83c18d626c264667b358e53e0b6fda63]) successfully deleted
I20251028 02:59:53.678094 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8a81c8bc182442bd8596251b99f90a2d
W20251028 02:59:53.678310 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 309a9d10178e4984a18e10dfd11cba22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 309a9d10178e4984a18e10dfd11cba22 already in progress: creating tablet
I20251028 02:59:53.678392 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 8a81c8bc182442bd8596251b99f90a2d deleted and replaced by tablet c1470187cdc74716b727052ea2b0b4a0
I20251028 02:59:53.678469 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 002de8aede994720b8163f41938af6c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.679343 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a81c8bc182442bd8596251b99f90a2d with delete_type TABLET_DATA_DELETED (replaced by tablet c1470187cdc74716b727052ea2b0b4a0) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.679558 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a81c8bc182442bd8596251b99f90a2d with delete_type TABLET_DATA_DELETED (replaced by tablet c1470187cdc74716b727052ea2b0b4a0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.681062 8551 tablet_replica.cc:333] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.681216 8551 raft_consensus.cc:2243] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.681341 8551 raft_consensus.cc:2272] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.681625 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a81c8bc182442bd8596251b99f90a2d with delete_type TABLET_DATA_DELETED (replaced by tablet c1470187cdc74716b727052ea2b0b4a0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.681766 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2f67b25b85194ce0be43f01adccd6c48
I20251028 02:59:53.681946 8545 tablet_bootstrap.cc:492] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.682700 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f67b25b85194ce0be43f01adccd6c48 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.682863 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f67b25b85194ce0be43f01adccd6c48 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.683259 8552 ts_tablet_manager.cc:1916] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.683262 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_0e8ca626159c41aeb8e360f6e038f7a3 [id=eb0e103661fc4bc7b27bb78fc333fd56]). Expected version 1 got 0
I20251028 02:59:53.683604 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 4726573f4b994b2591332c4a42fb10a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.683952 8551 ts_tablet_manager.cc:1916] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.685178 8259 tablet_service.cc:1505] Processing CreateTablet for tablet df60914093f444fda88c13a132587513 (DEFAULT_TABLE table=default.Table_0d1a3e0b91aa41dcb3264e04169fdd5e [id=45e7b7c657e54d43a490443857404c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.685192 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aeddf70e5bd342f19f3f427d1df73814" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.685420 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df60914093f444fda88c13a132587513. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:53.685590 8218 leader_election.cc:343] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.685663 8218 leader_election.cc:304] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:53.685742 8218 raft_consensus.cc:2686] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251028 02:59:53.685981 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 2f67b25b85194ce0be43f01adccd6c48 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.686736 8545 tablet_bootstrap.cc:654] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.687147 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 506396e55fc242ada23d66088984f0ae (DEFAULT_TABLE table=default.Table_ba1740929d404081b7451d23bc910475 [id=d41b990640fd4401a2d09310961cbba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.687392 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 506396e55fc242ada23d66088984f0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.688230 8552 ts_tablet_manager.cc:1929] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.688293 8552 log.cc:1199] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/aeddf70e5bd342f19f3f427d1df73814
I20251028 02:59:53.688601 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 506396e55fc242ada23d66088984f0ae (DEFAULT_TABLE table=default.Table_ba1740929d404081b7451d23bc910475 [id=d41b990640fd4401a2d09310961cbba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.688844 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 506396e55fc242ada23d66088984f0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.689211 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 72350c60d73b41829d99ac8d01aa04bc (DEFAULT_TABLE table=default.Table_f64a73f0705a4a119870d8d17497b338 [id=5de7688626854f0c8cabe8fb32b28ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.689456 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72350c60d73b41829d99ac8d01aa04bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.689527 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 9bf28d9bd3bd462f864fc4f20468e605 (DEFAULT_TABLE table=default.Table_ce36e2a0a4374085b1e87288b3288ab9 [id=e61fa366db844911bb80af844de41fa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.689585 8396 tablet_service.cc:1505] Processing CreateTablet for tablet df60914093f444fda88c13a132587513 (DEFAULT_TABLE table=default.Table_0d1a3e0b91aa41dcb3264e04169fdd5e [id=45e7b7c657e54d43a490443857404c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.689767 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bf28d9bd3bd462f864fc4f20468e605. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.689860 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df60914093f444fda88c13a132587513. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.691186 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 9bf28d9bd3bd462f864fc4f20468e605 (DEFAULT_TABLE table=default.Table_ce36e2a0a4374085b1e87288b3288ab9 [id=e61fa366db844911bb80af844de41fa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.691422 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bf28d9bd3bd462f864fc4f20468e605. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.693081 8116 tablet_service.cc:1505] Processing CreateTablet for tablet b6d868c81a7e402ba9305bb2aa90625b (DEFAULT_TABLE table=default.Table_3ed9eb1b206e4d69ae9e4113517486f9 [id=da79674aa8cd468e84114457ee46766e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.693297 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d868c81a7e402ba9305bb2aa90625b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.693560 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 9e260030c0e6423cba37a2e3af1c6400 (DEFAULT_TABLE table=default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.693745 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e260030c0e6423cba37a2e3af1c6400. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.694125 8262 tablet_service.cc:1505] Processing CreateTablet for tablet b6d868c81a7e402ba9305bb2aa90625b (DEFAULT_TABLE table=default.Table_3ed9eb1b206e4d69ae9e4113517486f9 [id=da79674aa8cd468e84114457ee46766e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.694340 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d868c81a7e402ba9305bb2aa90625b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.695356 8261 tablet_service.cc:1505] Processing CreateTablet for tablet bf7f6a7ada824eb1bca9be5344c9287e (DEFAULT_TABLE table=default.Table_fa500fb2e478468fac29b3a6682b2283 [id=6977afa2c1b2448c936af51bc6e23d36]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.695588 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7f6a7ada824eb1bca9be5344c9287e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.695720 9011 raft_consensus.cc:493] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.695794 9011 raft_consensus.cc:515] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.696182 9011 leader_election.cc:290] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.696463 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 72350c60d73b41829d99ac8d01aa04bc (DEFAULT_TABLE table=default.Table_f64a73f0705a4a119870d8d17497b338 [id=5de7688626854f0c8cabe8fb32b28ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.696507 8118 tablet_service.cc:1505] Processing CreateTablet for tablet bf7f6a7ada824eb1bca9be5344c9287e (DEFAULT_TABLE table=default.Table_fa500fb2e478468fac29b3a6682b2283 [id=6977afa2c1b2448c936af51bc6e23d36]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.696717 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72350c60d73b41829d99ac8d01aa04bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.696729 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7f6a7ada824eb1bca9be5344c9287e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.697312 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 9e260030c0e6423cba37a2e3af1c6400 (DEFAULT_TABLE table=default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.697527 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e260030c0e6423cba37a2e3af1c6400. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.697767 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 4416e1e10f53401cbddc525d9556d107 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.698271 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 9bf28d9bd3bd462f864fc4f20468e605 (DEFAULT_TABLE table=default.Table_ce36e2a0a4374085b1e87288b3288ab9 [id=e61fa366db844911bb80af844de41fa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.698467 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bf28d9bd3bd462f864fc4f20468e605. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.698781 8394 tablet_service.cc:1505] Processing CreateTablet for tablet bf7f6a7ada824eb1bca9be5344c9287e (DEFAULT_TABLE table=default.Table_fa500fb2e478468fac29b3a6682b2283 [id=6977afa2c1b2448c936af51bc6e23d36]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.698980 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7f6a7ada824eb1bca9be5344c9287e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:53.699458 8220 leader_election.cc:343] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: ab1ec08da9f64109a3fe9400fe4e366c
I20251028 02:59:53.699728 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 506396e55fc242ada23d66088984f0ae (DEFAULT_TABLE table=default.Table_ba1740929d404081b7451d23bc910475 [id=d41b990640fd4401a2d09310961cbba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.699939 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 506396e55fc242ada23d66088984f0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.702354 8552 ts_tablet_manager.cc:1950] T aeddf70e5bd342f19f3f427d1df73814 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.703445 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet aeddf70e5bd342f19f3f427d1df73814 (table default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]) successfully deleted
I20251028 02:59:53.704159 8546 tablet_bootstrap.cc:492] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.704253 8546 ts_tablet_manager.cc:1403] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.101s user 0.000s sys 0.002s
I20251028 02:59:53.704517 8551 ts_tablet_manager.cc:1929] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.704579 8551 log.cc:1199] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/6c87d03fad194ca89f1808da6dbdf0a4
I20251028 02:59:53.704770 8551 ts_tablet_manager.cc:1950] T 6c87d03fad194ca89f1808da6dbdf0a4 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.704775 8546 raft_consensus.cc:359] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.704918 8546 raft_consensus.cc:385] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.704952 8546 raft_consensus.cc:740] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.705044 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 72350c60d73b41829d99ac8d01aa04bc (DEFAULT_TABLE table=default.Table_f64a73f0705a4a119870d8d17497b338 [id=5de7688626854f0c8cabe8fb32b28ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.705086 8546 consensus_queue.cc:260] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.705298 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72350c60d73b41829d99ac8d01aa04bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.705406 8546 ts_tablet_manager.cc:1434] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20251028 02:59:53.705484 8218 leader_election.cc:343] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: ab1ec08da9f64109a3fe9400fe4e366c
I20251028 02:59:53.705554 8218 leader_election.cc:304] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.705616 8546 tablet_bootstrap.cc:492] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.705832 8556 raft_consensus.cc:2749] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251028 02:59:53.706367 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 8a81c8bc182442bd8596251b99f90a2d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a81c8bc182442bd8596251b99f90a2d already in progress: creating tablet
I20251028 02:59:53.706584 8546 tablet_bootstrap.cc:654] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.706704 8550 tablet_replica.cc:333] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.706878 8550 raft_consensus.cc:2243] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:53.707131 8550 raft_consensus.cc:2272] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.707554 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 9e260030c0e6423cba37a2e3af1c6400 (DEFAULT_TABLE table=default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.707608 8550 ts_tablet_manager.cc:1916] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.707784 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e260030c0e6423cba37a2e3af1c6400. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.707852 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 6c87d03fad194ca89f1808da6dbdf0a4 (table default.Table_6d8f283a536e46fcbf3d8abcf9d703bb [id=a6d84958fd654087940c9806e2e3fbc0]) successfully deleted
I20251028 02:59:53.708783 8616 consensus_queue.cc:1048] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.708979 8551 tablet_replica.cc:333] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.709100 8551 raft_consensus.cc:2243] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.709194 8551 raft_consensus.cc:2272] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.710026 8394 tablet_service.cc:1505] Processing CreateTablet for tablet b6d868c81a7e402ba9305bb2aa90625b (DEFAULT_TABLE table=default.Table_3ed9eb1b206e4d69ae9e4113517486f9 [id=da79674aa8cd468e84114457ee46766e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.710289 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d868c81a7e402ba9305bb2aa90625b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.711338 8551 ts_tablet_manager.cc:1916] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.713121 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet e4cdbf69eceb4bec9de3368ec959d5f0 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.713212 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_416a2058166f4d5a94419c7838d41b75" } modify_external_catalogs: true
I20251028 02:59:53.713325 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_416a2058166f4d5a94419c7838d41b75" } modify_external_catalogs: true
I20251028 02:59:53.713343 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6cad41d6014141378b584a34027d2caf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.713721 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cad41d6014141378b584a34027d2caf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.713848 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2d9658afb66c44639064e40eaf4fa075" } new_table_name: "default.Table_2c98fdd4e565487c85c1d9128125e212" modify_external_catalogs: true
I20251028 02:59:53.714033 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b9407dac1631402082afb2b12dc61dbd" } new_table_name: "default.Table_bfd3f08061c3433f878612643c04387b" modify_external_catalogs: true
I20251028 02:59:53.714648 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_806d85836a4c44a986095f5108a93d7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.714967 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_152433eb6e11487198b81047f13cc1f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.715291 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_152433eb6e11487198b81047f13cc1f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.715719 8551 ts_tablet_manager.cc:1929] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.715806 8551 log.cc:1199] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/002de8aede994720b8163f41938af6c4
W20251028 02:59:53.715018 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_806d85836a4c44a986095f5108a93d7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.716043 8551 ts_tablet_manager.cc:1950] T 002de8aede994720b8163f41938af6c4 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.716712 8552 tablet_replica.cc:333] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.716948 8552 raft_consensus.cc:2243] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.717146 8552 raft_consensus.cc:2272] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.717759 8552 ts_tablet_manager.cc:1916] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.717814 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 002de8aede994720b8163f41938af6c4 (table default.Table_c3d6504130c54bdeade5bb23b6cd0679 [id=371c4929d59942f79f41ad071b61e017]) successfully deleted
W20251028 02:59:53.718279 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 8a81c8bc182442bd8596251b99f90a2d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a81c8bc182442bd8596251b99f90a2d already in progress: creating tablet
I20251028 02:59:53.719161 8551 tablet_replica.cc:333] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.719836 8550 ts_tablet_manager.cc:1929] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.719321 8551 raft_consensus.cc:2243] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.720780 8544 tablet_bootstrap.cc:492] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.720992 8551 raft_consensus.cc:2272] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.721035 8544 ts_tablet_manager.cc:1403] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.094s user 0.002s sys 0.000s
I20251028 02:59:53.721503 8544 raft_consensus.cc:359] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.721596 8544 raft_consensus.cc:385] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.721596 8551 ts_tablet_manager.cc:1916] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.721623 8544 raft_consensus.cc:740] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.721735 8544 consensus_queue.cc:260] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.722011 8544 ts_tablet_manager.cc:1434] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.722179 8544 tablet_bootstrap.cc:492] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.723137 8544 tablet_bootstrap.cc:654] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.727495 8550 log.cc:1199] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2f67b25b85194ce0be43f01adccd6c48
W20251028 02:59:53.727653 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4bcdf0368e394d649086042f1fe9774b consensus state has no leader
I20251028 02:59:53.727938 8550 ts_tablet_manager.cc:1950] T 2f67b25b85194ce0be43f01adccd6c48 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 02:59:53.727977 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4ff8c974231f4b8aa18c18a05ad3e738 consensus state has no leader
I20251028 02:59:53.730373 8549 consensus_queue.cc:1048] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.731746 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2f67b25b85194ce0be43f01adccd6c48 (table default.Table_153da548190a42a78cad7ae8553d6fba [id=9f4f24d46fc048909a3907301bb526de]) successfully deleted
I20251028 02:59:53.731807 8545 tablet_bootstrap.cc:492] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.731906 8545 ts_tablet_manager.cc:1403] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.000s
I20251028 02:59:53.732106 8552 ts_tablet_manager.cc:1929] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.732227 8552 log.cc:1199] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/50a0a3d690154de7acd55f37ab736d13
I20251028 02:59:53.732491 8552 ts_tablet_manager.cc:1950] T 50a0a3d690154de7acd55f37ab736d13 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.732445 8545 raft_consensus.cc:359] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.732575 8545 raft_consensus.cc:385] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.732627 8545 raft_consensus.cc:740] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.732755 8545 consensus_queue.cc:260] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.733055 8545 ts_tablet_manager.cc:1434] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.733261 8545 tablet_bootstrap.cc:492] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.733457 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 50a0a3d690154de7acd55f37ab736d13 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]) successfully deleted
W20251028 02:59:53.733897 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 8a81c8bc182442bd8596251b99f90a2d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a81c8bc182442bd8596251b99f90a2d already in progress: creating tablet
I20251028 02:59:53.734128 8552 tablet_replica.cc:333] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.734292 8552 raft_consensus.cc:2243] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.734354 8545 tablet_bootstrap.cc:654] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.734470 8552 raft_consensus.cc:2272] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.734866 8552 ts_tablet_manager.cc:1916] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.738121 8544 tablet_bootstrap.cc:492] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.738222 8544 ts_tablet_manager.cc:1403] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 02:59:53.738659 8544 raft_consensus.cc:359] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.738755 8544 raft_consensus.cc:385] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.738793 8544 raft_consensus.cc:740] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.738902 8544 consensus_queue.cc:260] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.739147 8544 ts_tablet_manager.cc:1434] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.739369 8544 tablet_bootstrap.cc:492] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.740175 8546 tablet_bootstrap.cc:492] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.740262 8546 ts_tablet_manager.cc:1403] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.001s
I20251028 02:59:53.740365 8544 tablet_bootstrap.cc:654] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.740759 8546 raft_consensus.cc:359] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.740869 8546 raft_consensus.cc:385] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.740904 8546 raft_consensus.cc:740] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.741014 8546 consensus_queue.cc:260] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.741286 8546 ts_tablet_manager.cc:1434] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.741461 8546 tablet_bootstrap.cc:492] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.743057 8546 tablet_bootstrap.cc:654] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.743449 8552 ts_tablet_manager.cc:1929] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.744019 8552 log.cc:1199] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/2f67b25b85194ce0be43f01adccd6c48
I20251028 02:59:53.744345 8552 ts_tablet_manager.cc:1950] T 2f67b25b85194ce0be43f01adccd6c48 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.745620 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 2f67b25b85194ce0be43f01adccd6c48 (table default.Table_153da548190a42a78cad7ae8553d6fba [id=9f4f24d46fc048909a3907301bb526de]) successfully deleted
I20251028 02:59:53.746251 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet db6eadc5927d4876af9a50b44be97a63 with delete_type TABLET_DATA_DELETED (replaced by tablet 67c6ceefa3294af8b68150609f46c117) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.746408 8544 tablet_bootstrap.cc:492] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.746491 8544 ts_tablet_manager.cc:1403] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:53.746933 8544 raft_consensus.cc:359] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.747037 8544 raft_consensus.cc:385] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.747066 8544 raft_consensus.cc:740] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.747169 8544 consensus_queue.cc:260] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.747455 8544 ts_tablet_manager.cc:1434] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.747622 8544 tablet_bootstrap.cc:492] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.748719 8544 tablet_bootstrap.cc:654] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.750715 8550 tablet_replica.cc:333] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.750912 8550 raft_consensus.cc:2243] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.751056 8550 raft_consensus.cc:2272] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.751555 8550 ts_tablet_manager.cc:1916] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.753090 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet aeddf70e5bd342f19f3f427d1df73814 with delete_type TABLET_DATA_DELETED (replaced by tablet 5fd9cc1847ff4df7a09e2e698786d515) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.753806 8551 ts_tablet_manager.cc:1929] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:53.754304 8551 log.cc:1199] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2f67b25b85194ce0be43f01adccd6c48
I20251028 02:59:53.754355 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 7801d57550dc40c39f2d3cf7e11e3713
I20251028 02:59:53.754649 8551 ts_tablet_manager.cc:1950] T 2f67b25b85194ce0be43f01adccd6c48 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.755429 8551 tablet_replica.cc:333] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.755573 8551 raft_consensus.cc:2243] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.755640 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2f67b25b85194ce0be43f01adccd6c48 (table default.Table_153da548190a42a78cad7ae8553d6fba [id=9f4f24d46fc048909a3907301bb526de]) successfully deleted
I20251028 02:59:53.755699 8551 raft_consensus.cc:2272] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.755697 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 7801d57550dc40c39f2d3cf7e11e3713 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.755870 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 7801d57550dc40c39f2d3cf7e11e3713 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.756138 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 7801d57550dc40c39f2d3cf7e11e3713 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.756709 8551 ts_tablet_manager.cc:1916] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.756994 8545 tablet_bootstrap.cc:492] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.757099 8545 ts_tablet_manager.cc:1403] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20251028 02:59:53.757200 8550 ts_tablet_manager.cc:1929] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.757340 8550 log.cc:1199] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/db6eadc5927d4876af9a50b44be97a63
I20251028 02:59:53.757627 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet df10bbbd757a4da8905718b61cbb10c8 (table default.Table_bb39271bc8354db88ef788080eb177bc [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 3 got 0
I20251028 02:59:53.757692 8545 raft_consensus.cc:359] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.757791 8545 raft_consensus.cc:385] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.757833 8545 raft_consensus.cc:740] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.757920 8544 tablet_bootstrap.cc:492] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.757997 8544 ts_tablet_manager.cc:1403] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:53.758621 8544 raft_consensus.cc:359] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.758726 8544 raft_consensus.cc:385] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.758754 8544 raft_consensus.cc:740] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.758858 8545 consensus_queue.cc:260] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.759469 8551 ts_tablet_manager.cc:1929] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:53.759538 8551 log.cc:1199] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4726573f4b994b2591332c4a42fb10a1
I20251028 02:59:53.759773 8551 ts_tablet_manager.cc:1950] T 4726573f4b994b2591332c4a42fb10a1 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.760427 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4726573f4b994b2591332c4a42fb10a1 (table default.Table_83d3bc102e114c3eb1bb7e3c34291f8c [id=b0eb2947677348c9a4f7128df96afbbf]) successfully deleted
W20251028 02:59:53.760665 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 7801d57550dc40c39f2d3cf7e11e3713 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7801d57550dc40c39f2d3cf7e11e3713 already in progress: creating tablet
I20251028 02:59:53.760207 8544 consensus_queue.cc:260] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.762774 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 929cf41ac7b0437387fbc488983f5cac (DEFAULT_TABLE table=default.Table_aa25a85ac8d24940a727a586d6c630db [id=10f40fa1652f495e91fcbd5852bc3464]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.762915 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]). Expected version 1 got 0
I20251028 02:59:53.763087 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 929cf41ac7b0437387fbc488983f5cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.764217 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f4eb60b8380645758ce2cadfaff1330a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.764264 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 2bd4558fc315491687a12efce7ee8ab2 (DEFAULT_TABLE table=default.Table_f2fce0c2e0b348e18d259ef0ee58e114 [id=a8812c2764af480bbc45bf74644d4fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.764264 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 929cf41ac7b0437387fbc488983f5cac (DEFAULT_TABLE table=default.Table_aa25a85ac8d24940a727a586d6c630db [id=10f40fa1652f495e91fcbd5852bc3464]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.764494 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd4558fc315491687a12efce7ee8ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.764494 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 929cf41ac7b0437387fbc488983f5cac. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:53.764588 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4eb60b8380645758ce2cadfaff1330a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.767966 8546 tablet_bootstrap.cc:492] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.768079 8546 ts_tablet_manager.cc:1403] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.001s
I20251028 02:59:53.768574 8546 raft_consensus.cc:359] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.768697 8546 raft_consensus.cc:385] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.768730 8546 raft_consensus.cc:740] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.768870 8546 consensus_queue.cc:260] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.769834 8545 ts_tablet_manager.cc:1434] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.013s user 0.001s sys 0.000s
I20251028 02:59:53.770787 8546 ts_tablet_manager.cc:1434] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:53.771966 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet a923402bbc734a0cb44aaf590d1e68ff with delete_type TABLET_DATA_DELETED (replaced by tablet 9e260030c0e6423cba37a2e3af1c6400) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.772022 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 929cf41ac7b0437387fbc488983f5cac (DEFAULT_TABLE table=default.Table_aa25a85ac8d24940a727a586d6c630db [id=10f40fa1652f495e91fcbd5852bc3464]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.772218 8545 tablet_bootstrap.cc:492] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.772321 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 929cf41ac7b0437387fbc488983f5cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.772360 8546 tablet_bootstrap.cc:492] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.774030 8546 tablet_bootstrap.cc:654] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.774546 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_75e654d01f4147b3b7d928c99967c5cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:53.774660 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 2bd4558fc315491687a12efce7ee8ab2 (DEFAULT_TABLE table=default.Table_f2fce0c2e0b348e18d259ef0ee58e114 [id=a8812c2764af480bbc45bf74644d4fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.774873 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd4558fc315491687a12efce7ee8ab2. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:53.774978 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75e654d01f4147b3b7d928c99967c5cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.773391 8545 tablet_bootstrap.cc:654] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.775889 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2fd19ddb9673469a98456a4520898183"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.776237 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2fd19ddb9673469a98456a4520898183 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.776919 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (table default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]). Expected version 1 got 0
I20251028 02:59:53.776903 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 2bd4558fc315491687a12efce7ee8ab2 (DEFAULT_TABLE table=default.Table_f2fce0c2e0b348e18d259ef0ee58e114 [id=a8812c2764af480bbc45bf74644d4fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.777187 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd4558fc315491687a12efce7ee8ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.777688 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a1cf1772f69f4e4284a15ea70a3407d6" } new_table_name: "default.Table_f73fbea5f42f4caa815469fbb7504f49" modify_external_catalogs: true
I20251028 02:59:53.778160 8544 ts_tablet_manager.cc:1434] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.020s user 0.001s sys 0.000s
I20251028 02:59:53.778460 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_33d362b6df3e40b68a3221d89a6b8358" } modify_external_catalogs: true
I20251028 02:59:53.778548 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_33d362b6df3e40b68a3221d89a6b8358" } modify_external_catalogs: true
I20251028 02:59:53.779261 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ab9ea7fcee93462394ebd14c2006d4e5" } new_table_name: "default.Table_26410f48ded64d1285940aeff4f1fc07" modify_external_catalogs: true
I20251028 02:59:53.779901 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet aec9427ed4284d66b9bd46c0e1c8774c with delete_type TABLET_DATA_DELETED (replaced by tablet de38b2d54cec4f4e892047f4650f611b) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.780797 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]). Expected version 1 got 0
I20251028 02:59:53.781220 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_632504e5f4754d1582584291b0702d3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.781555 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_632504e5f4754d1582584291b0702d3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.782042 8544 tablet_bootstrap.cc:492] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.782441 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ad3466e031ae40059b8e3ec01c7ed5a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.782842 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad3466e031ae40059b8e3ec01c7ed5a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.783584 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b1c390b850f444aca6a3856952225164"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.784004 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b1c390b850f444aca6a3856952225164 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.784180 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 2a2ebe28dbd2439e963319ec6e515635 (DEFAULT_TABLE table=default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.784449 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a2ebe28dbd2439e963319ec6e515635. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.786595 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 35e211a1a8c547baaee1199f7103bfad (DEFAULT_TABLE table=default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.786857 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e211a1a8c547baaee1199f7103bfad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.787179 8396 tablet_service.cc:1505] Processing CreateTablet for tablet c1470187cdc74716b727052ea2b0b4a0 (DEFAULT_TABLE table=default.Table_63b74aebe1364823bb6a10f4a8942db8 [id=485d972ab1864e26ace68ba52bb671ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.787492 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1470187cdc74716b727052ea2b0b4a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.788203 8549 raft_consensus.cc:493] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.788264 8117 tablet_service.cc:1505] Processing CreateTablet for tablet c1470187cdc74716b727052ea2b0b4a0 (DEFAULT_TABLE table=default.Table_63b74aebe1364823bb6a10f4a8942db8 [id=485d972ab1864e26ace68ba52bb671ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.788293 8549 raft_consensus.cc:515] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.788486 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1470187cdc74716b727052ea2b0b4a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.788604 8549 leader_election.cc:290] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.789129 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 2a2ebe28dbd2439e963319ec6e515635 (DEFAULT_TABLE table=default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.789454 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 2a2ebe28dbd2439e963319ec6e515635 (DEFAULT_TABLE table=default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.789722 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a2ebe28dbd2439e963319ec6e515635. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.790349 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ca4ff164b24b7498f953ec11cf46be" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:53.790736 8356 leader_election.cc:343] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.790781 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 35e211a1a8c547baaee1199f7103bfad (DEFAULT_TABLE table=default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.791356 8544 tablet_bootstrap.cc:654] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.791538 8549 raft_consensus.cc:493] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.791605 8549 raft_consensus.cc:515] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.791956 8549 leader_election.cc:290] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.791982 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ca4ff164b24b7498f953ec11cf46be" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.792109 8258 tablet_service.cc:1505] Processing CreateTablet for tablet c1470187cdc74716b727052ea2b0b4a0 (DEFAULT_TABLE table=default.Table_63b74aebe1364823bb6a10f4a8942db8 [id=485d972ab1864e26ace68ba52bb671ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.792174 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 35e211a1a8c547baaee1199f7103bfad (DEFAULT_TABLE table=default.Table_ab9ea7fcee93462394ebd14c2006d4e5 [id=3191deea26744b009e341282c1babcbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.792377 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a2ebe28dbd2439e963319ec6e515635. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:53.793015 8354 leader_election.cc:343] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.793085 8354 leader_election.cc:304] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.793207 8549 raft_consensus.cc:2749] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.757565 8550 ts_tablet_manager.cc:1950] T db6eadc5927d4876af9a50b44be97a63 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.793416 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03959a61996b40d8b9ef8b4b9708f017" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:53.793730 8354 leader_election.cc:343] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.794310 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "03959a61996b40d8b9ef8b4b9708f017" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:53.794636 8356 leader_election.cc:343] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.794696 8356 leader_election.cc:304] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.794827 8549 raft_consensus.cc:2749] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.792145 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e211a1a8c547baaee1199f7103bfad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.795634 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet db6eadc5927d4876af9a50b44be97a63 (table default.Table_12396134528e4544a80934aa91d07353 [id=86c2f59df3dd4d2195d18835dceff79b]) successfully deleted
I20251028 02:59:53.796195 8550 tablet_replica.cc:333] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.796339 8550 raft_consensus.cc:2243] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.796450 8550 raft_consensus.cc:2272] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.796881 8552 tablet_replica.cc:333] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.796941 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet e4cdbf69eceb4bec9de3368ec959d5f0
I20251028 02:59:53.797029 8552 raft_consensus.cc:2243] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.797240 7969 catalog_manager.cc:6719] ReplaceTablet: tablet e4cdbf69eceb4bec9de3368ec959d5f0 deleted and replaced by tablet 74120decd53d4de19c5f6fedf70ecaae
I20251028 02:59:53.797318 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1470187cdc74716b727052ea2b0b4a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.797425 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet e4cdbf69eceb4bec9de3368ec959d5f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 74120decd53d4de19c5f6fedf70ecaae) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.797951 8545 tablet_bootstrap.cc:492] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.798058 8545 ts_tablet_manager.cc:1403] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.002s
W20251028 02:59:53.798738 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 7801d57550dc40c39f2d3cf7e11e3713 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7801d57550dc40c39f2d3cf7e11e3713 already in progress: creating tablet
I20251028 02:59:53.798828 8545 raft_consensus.cc:359] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.799739 8545 raft_consensus.cc:385] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.799785 8545 raft_consensus.cc:740] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.799890 8545 consensus_queue.cc:260] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.800612 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e211a1a8c547baaee1199f7103bfad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.801040 8545 ts_tablet_manager.cc:1434] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251028 02:59:53.801467 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_88e852b7e3584db7bfcfd5175978ade6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.801898 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88e852b7e3584db7bfcfd5175978ade6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.802711 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet e4cdbf69eceb4bec9de3368ec959d5f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 74120decd53d4de19c5f6fedf70ecaae) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.802706 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet e4cdbf69eceb4bec9de3368ec959d5f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 74120decd53d4de19c5f6fedf70ecaae) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.804593 8550 ts_tablet_manager.cc:1916] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.804817 8552 raft_consensus.cc:2272] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.807863 8552 ts_tablet_manager.cc:1916] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.810134 8546 tablet_bootstrap.cc:492] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.810236 8546 ts_tablet_manager.cc:1403] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.000s
I20251028 02:59:53.810766 8546 raft_consensus.cc:359] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.810894 8546 raft_consensus.cc:385] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.810931 8546 raft_consensus.cc:740] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.811065 8546 consensus_queue.cc:260] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.811373 8546 ts_tablet_manager.cc:1434] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:53.811589 8546 tablet_bootstrap.cc:492] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.813946 8545 tablet_bootstrap.cc:492] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:53.816462 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a8331b21af69463cb7589619e784051d consensus state has no leader
I20251028 02:59:53.816497 8545 tablet_bootstrap.cc:654] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.819183 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet eaa41f8bf7c54673b65893acc59052c1
I20251028 02:59:53.819875 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet eaa41f8bf7c54673b65893acc59052c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:53.820150 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet e4cdbf69eceb4bec9de3368ec959d5f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e4cdbf69eceb4bec9de3368ec959d5f0 already in progress: creating tablet
I20251028 02:59:53.820482 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 1ef353f5e715452eae1f1c191fe79d1d from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:53.820685 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 35e211a1a8c547baaee1199f7103bfad consensus state has no leader
I20251028 02:59:53.820900 9011 raft_consensus.cc:493] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.820998 9011 raft_consensus.cc:515] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.821370 9011 leader_election.cc:290] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.822045 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "df10bbbd757a4da8905718b61cbb10c8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.822176 8420 raft_consensus.cc:2468] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:53.822916 8220 leader_election.cc:304] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.823076 9011 raft_consensus.cc:2804] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.823124 9011 raft_consensus.cc:493] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.823161 9011 raft_consensus.cc:3060] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.823354 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8cae8460736e4dcb9108757c62255907" } modify_external_catalogs: true
I20251028 02:59:53.823455 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8cae8460736e4dcb9108757c62255907" } modify_external_catalogs: true
I20251028 02:59:53.824353 9011 raft_consensus.cc:515] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.824707 9011 leader_election.cc:290] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.825171 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "df10bbbd757a4da8905718b61cbb10c8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:53.825291 8420 raft_consensus.cc:3060] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.825476 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet eaa41f8bf7c54673b65893acc59052c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.825814 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet eaa41f8bf7c54673b65893acc59052c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.826050 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "df10bbbd757a4da8905718b61cbb10c8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:53.826316 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "df10bbbd757a4da8905718b61cbb10c8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.826347 8420 raft_consensus.cc:2468] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:53.826627 8220 leader_election.cc:304] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.826730 9011 raft_consensus.cc:493] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.826817 8556 raft_consensus.cc:2804] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:53.826794 9011 raft_consensus.cc:515] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.826880 8556 raft_consensus.cc:697] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:53.826997 8556 consensus_queue.cc:237] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.827159 9011 leader_election.cc:290] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.827589 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37dcf605a2984be1817d9621e4a41448" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.827709 8420 raft_consensus.cc:2468] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:53.827983 8220 leader_election.cc:304] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.828136 8965 raft_consensus.cc:2804] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.828184 8965 raft_consensus.cc:493] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.828220 8965 raft_consensus.cc:3060] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.828368 9144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:53.828470 9144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: This master's current role is: LEADER
I20251028 02:59:53.829264 8965 raft_consensus.cc:515] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.829591 8965 leader_election.cc:290] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.830616 8552 ts_tablet_manager.cc:1929] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.830687 8552 log.cc:1199] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/a923402bbc734a0cb44aaf590d1e68ff
I20251028 02:59:53.831560 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37dcf605a2984be1817d9621e4a41448" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.831563 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37dcf605a2984be1817d9621e4a41448" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:53.833325 8546 tablet_bootstrap.cc:654] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:53.834156 8218 leader_election.cc:343] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.834254 8550 ts_tablet_manager.cc:1929] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.834326 8550 log.cc:1199] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/aeddf70e5bd342f19f3f427d1df73814
W20251028 02:59:53.834354 8218 leader_election.cc:343] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.834534 8550 ts_tablet_manager.cc:1950] T aeddf70e5bd342f19f3f427d1df73814 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 02:59:53.835273 8218 leader_election.cc:343] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:53.835510 8218 leader_election.cc:343] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.835484 7962 catalog_manager.cc:5649] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:53.835677 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet df10bbbd757a4da8905718b61cbb10c8 (table default.Table_bb39271bc8354db88ef788080eb177bc [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 3 got 0
W20251028 02:59:53.835959 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet eaa41f8bf7c54673b65893acc59052c1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eaa41f8bf7c54673b65893acc59052c1 already in progress: creating tablet
I20251028 02:59:53.836097 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet edecfb891eaa4a718ef1fdf399e7ef9d with delete_type TABLET_DATA_DELETED (replaced by tablet 7801d57550dc40c39f2d3cf7e11e3713) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.836231 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 12dce4bb3d4b43608d7da9350b14e1d0 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.836270 8551 tablet_replica.cc:333] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.836315 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_33402cf4a2024ce3b5e0f7471e82322c" } new_table_name: "default.Table_e4f1369ad6ad4110b5854544d4a266bd" modify_external_catalogs: true
I20251028 02:59:53.836406 8551 raft_consensus.cc:2243] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.836521 8551 raft_consensus.cc:2272] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.836762 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 02f6a32db68e4c609da39c3ee6c97348 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.836942 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "37dcf605a2984be1817d9621e4a41448" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:53.837013 8551 ts_tablet_manager.cc:1916] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.837059 8420 raft_consensus.cc:3060] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.837615 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet aeddf70e5bd342f19f3f427d1df73814 (table default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]) successfully deleted
I20251028 02:59:53.838038 8420 raft_consensus.cc:2468] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:53.838320 8544 tablet_bootstrap.cc:492] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.838419 8544 ts_tablet_manager.cc:1403] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.000s
I20251028 02:59:53.838886 8544 raft_consensus.cc:359] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.839078 8551 ts_tablet_manager.cc:1929] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.839154 8551 log.cc:1199] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/edecfb891eaa4a718ef1fdf399e7ef9d
I20251028 02:59:53.839524 8220 leader_election.cc:304] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.839830 8551 ts_tablet_manager.cc:1950] T edecfb891eaa4a718ef1fdf399e7ef9d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.840063 9011 raft_consensus.cc:2804] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:53.840214 9011 raft_consensus.cc:697] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:53.840782 9011 consensus_queue.cc:237] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.841485 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet edecfb891eaa4a718ef1fdf399e7ef9d (table default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]) successfully deleted
I20251028 02:59:53.841635 8544 raft_consensus.cc:385] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.841692 8544 raft_consensus.cc:740] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.841940 8544 consensus_queue.cc:260] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.842233 8544 ts_tablet_manager.cc:1434] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:53.842406 8544 tablet_bootstrap.cc:492] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.844971 8544 tablet_bootstrap.cc:654] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:53.845860 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 7801d57550dc40c39f2d3cf7e11e3713 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7801d57550dc40c39f2d3cf7e11e3713 already in progress: creating tablet
W20251028 02:59:53.846279 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e4cdbf69eceb4bec9de3368ec959d5f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e4cdbf69eceb4bec9de3368ec959d5f0 already in progress: creating tablet
I20251028 02:59:53.846524 8552 ts_tablet_manager.cc:1950] T a923402bbc734a0cb44aaf590d1e68ff P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 02:59:53.846638 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet eaa41f8bf7c54673b65893acc59052c1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eaa41f8bf7c54673b65893acc59052c1 already in progress: creating tablet
I20251028 02:59:53.847457 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9feee73521df42ae89f5af5c5eacf9bd
I20251028 02:59:53.847563 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet df10bbbd757a4da8905718b61cbb10c8 (table default.Table_bb39271bc8354db88ef788080eb177bc [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 3 got 0
I20251028 02:59:53.847715 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet a923402bbc734a0cb44aaf590d1e68ff (table default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]) successfully deleted
I20251028 02:59:53.848516 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 9feee73521df42ae89f5af5c5eacf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.848699 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 9feee73521df42ae89f5af5c5eacf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.848773 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 9feee73521df42ae89f5af5c5eacf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.848804 7935 master_runner.cc:186] Error getting master registration for 127.7.154.189:45473: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111),
I20251028 02:59:53.848912 7935 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20251028 02:59:53.848973 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 9feee73521df42ae89f5af5c5eacf9bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9feee73521df42ae89f5af5c5eacf9bd already in progress: creating tablet
W20251028 02:59:53.849926 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9feee73521df42ae89f5af5c5eacf9bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9feee73521df42ae89f5af5c5eacf9bd already in progress: creating tablet
I20251028 02:59:53.853564 8544 tablet_bootstrap.cc:492] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.853677 8544 ts_tablet_manager.cc:1403] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:53.854163 8144 raft_consensus.cc:3060] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.854260 8285 raft_consensus.cc:1275] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:53.854238 8544 raft_consensus.cc:359] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.854339 8544 raft_consensus.cc:385] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.854375 8544 raft_consensus.cc:740] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.854496 8544 consensus_queue.cc:260] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.854627 8549 consensus_queue.cc:1048] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.854789 8544 ts_tablet_manager.cc:1434] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:53.855005 8544 tablet_bootstrap.cc:492] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.855258 8144 raft_consensus.cc:1275] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:53.856200 8544 tablet_bootstrap.cc:654] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.858230 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 50a0a3d690154de7acd55f37ab736d13 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ff8c974231f4b8aa18c18a05ad3e738) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.858220 8549 consensus_queue.cc:1048] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.858546 8550 tablet_replica.cc:333] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.858675 8550 raft_consensus.cc:2243] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.858764 8550 raft_consensus.cc:2272] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.859328 8550 ts_tablet_manager.cc:1916] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.859694 8552 tablet_replica.cc:333] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.859891 8552 raft_consensus.cc:2243] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.860224 8552 raft_consensus.cc:2272] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.860699 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 7e347497892b4add9f7b4e19bde67878 (table default.Table_e4f1369ad6ad4110b5854544d4a266bd [id=ccea14281b2c420a99cede953a8de44d]). Expected version 1 got 0
I20251028 02:59:53.862114 8546 tablet_bootstrap.cc:492] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.862212 8546 ts_tablet_manager.cc:1403] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.000s
I20251028 02:59:53.862751 8546 raft_consensus.cc:359] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.862880 8546 raft_consensus.cc:385] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.862891 8545 tablet_bootstrap.cc:492] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.862923 8546 raft_consensus.cc:740] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.862978 8545 ts_tablet_manager.cc:1403] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.001s
I20251028 02:59:53.863063 8546 consensus_queue.cc:260] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.863409 8546 ts_tablet_manager.cc:1434] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.863530 8545 raft_consensus.cc:359] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.863629 8546 tablet_bootstrap.cc:492] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.863641 8545 raft_consensus.cc:385] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.863679 8545 raft_consensus.cc:740] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.863803 8545 consensus_queue.cc:260] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.864102 8545 ts_tablet_manager.cc:1434] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:53.864348 8545 tablet_bootstrap.cc:492] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.865412 8545 tablet_bootstrap.cc:654] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.866410 9048 tablet.cc:1722] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:53.866814 8544 tablet_bootstrap.cc:492] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.866900 8544 ts_tablet_manager.cc:1403] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:53.867664 8544 raft_consensus.cc:359] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.867784 8544 raft_consensus.cc:385] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.867821 8544 raft_consensus.cc:740] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.867974 8544 consensus_queue.cc:260] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.868278 8544 ts_tablet_manager.cc:1434] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.868484 8544 tablet_bootstrap.cc:492] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.868650 9122 tablet.cc:1722] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:53.868949 8546 tablet_bootstrap.cc:654] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.869530 8544 tablet_bootstrap.cc:654] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.870182 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_23ea93791ab84db8acf0b4e6c21141d8" } modify_external_catalogs: true
I20251028 02:59:53.870318 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_23ea93791ab84db8acf0b4e6c21141d8" } modify_external_catalogs: true
I20251028 02:59:53.871359 8550 ts_tablet_manager.cc:1929] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.871483 8550 log.cc:1199] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/50a0a3d690154de7acd55f37ab736d13
I20251028 02:59:53.871753 8550 ts_tablet_manager.cc:1950] T 50a0a3d690154de7acd55f37ab736d13 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.874392 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 50a0a3d690154de7acd55f37ab736d13 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]) successfully deleted
I20251028 02:59:53.877384 7960 catalog_manager.cc:5649] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:53.877604 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 37dcf605a2984be1817d9621e4a41448 (table default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc]). Expected version 1 got 0
I20251028 02:59:53.878618 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 369397a89baa472285b25acd6c78e7ec (DEFAULT_TABLE table=default.Table_6cad41d6014141378b584a34027d2caf [id=8c5f437ae7374d68b232dbb3e43cc8aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.878931 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 369397a89baa472285b25acd6c78e7ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.879640 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 551aea93ba064bc89ae00b002eb7ccfc (DEFAULT_TABLE table=default.Table_806d85836a4c44a986095f5108a93d7f [id=c098c2727dd147e69471353bd9a741fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.879900 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 551aea93ba064bc89ae00b002eb7ccfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.879875 8552 ts_tablet_manager.cc:1916] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.880409 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 369397a89baa472285b25acd6c78e7ec (DEFAULT_TABLE table=default.Table_6cad41d6014141378b584a34027d2caf [id=8c5f437ae7374d68b232dbb3e43cc8aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.880609 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 369397a89baa472285b25acd6c78e7ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.881386 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 26a344e2e03e4eeaa688275e9ecbf63f (DEFAULT_TABLE table=default.Table_152433eb6e11487198b81047f13cc1f1 [id=8642a2095b8049de90e6df3a8570d730]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.881582 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a344e2e03e4eeaa688275e9ecbf63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.882869 8546 tablet_bootstrap.cc:492] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.882958 8546 ts_tablet_manager.cc:1403] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20251028 02:59:53.883545 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet da1563d65b4d4ab48325a4f08ee89ed5 (table default.Table_153d49d056a3458a9565c70d954187ae [id=ae8a178b437f4df4af15bb82bd7a95ee]). Expected version 1 got 0
I20251028 02:59:53.883507 8546 raft_consensus.cc:359] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.883618 8546 raft_consensus.cc:385] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.883628 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 369397a89baa472285b25acd6c78e7ec (DEFAULT_TABLE table=default.Table_6cad41d6014141378b584a34027d2caf [id=8c5f437ae7374d68b232dbb3e43cc8aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.883893 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 369397a89baa472285b25acd6c78e7ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.886663 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 26a344e2e03e4eeaa688275e9ecbf63f (DEFAULT_TABLE table=default.Table_152433eb6e11487198b81047f13cc1f1 [id=8642a2095b8049de90e6df3a8570d730]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.886945 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a344e2e03e4eeaa688275e9ecbf63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.886968 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5da94c10f01846a59e4ce4ff530330d3" } modify_external_catalogs: true
I20251028 02:59:53.887101 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5da94c10f01846a59e4ce4ff530330d3" } modify_external_catalogs: true
I20251028 02:59:53.887375 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet da1563d65b4d4ab48325a4f08ee89ed5 (table default.Table_153d49d056a3458a9565c70d954187ae [id=ae8a178b437f4df4af15bb82bd7a95ee]). Expected version 1 got 0
I20251028 02:59:53.889119 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 551aea93ba064bc89ae00b002eb7ccfc (DEFAULT_TABLE table=default.Table_806d85836a4c44a986095f5108a93d7f [id=c098c2727dd147e69471353bd9a741fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.889351 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 551aea93ba064bc89ae00b002eb7ccfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.889612 9023 tablet.cc:1722] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:53.892134 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet e4cdbf69eceb4bec9de3368ec959d5f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 74120decd53d4de19c5f6fedf70ecaae) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.892241 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 26a344e2e03e4eeaa688275e9ecbf63f (DEFAULT_TABLE table=default.Table_152433eb6e11487198b81047f13cc1f1 [id=8642a2095b8049de90e6df3a8570d730]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.892326 8551 tablet_replica.cc:333] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.892460 8551 raft_consensus.cc:2243] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.892529 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a344e2e03e4eeaa688275e9ecbf63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.892572 8551 raft_consensus.cc:2272] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.893052 8551 ts_tablet_manager.cc:1916] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.883651 8546 raft_consensus.cc:740] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.894081 8546 consensus_queue.cc:260] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.894879 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 551aea93ba064bc89ae00b002eb7ccfc (DEFAULT_TABLE table=default.Table_806d85836a4c44a986095f5108a93d7f [id=c098c2727dd147e69471353bd9a741fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.895128 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 551aea93ba064bc89ae00b002eb7ccfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.895380 8546 ts_tablet_manager.cc:1434] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:53.895569 8546 tablet_bootstrap.cc:492] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.896569 8546 tablet_bootstrap.cc:654] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.897431 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1ef353f5e715452eae1f1c191fe79d1d
I20251028 02:59:53.897725 7964 catalog_manager.cc:6719] ReplaceTablet: tablet 1ef353f5e715452eae1f1c191fe79d1d deleted and replaced by tablet 9a290b8114844aa89b33c75fffc8e1e7
I20251028 02:59:53.898871 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8ae2a510d64243a9b0fb7784ee56d29f
I20251028 02:59:53.898895 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 1ef353f5e715452eae1f1c191fe79d1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9a290b8114844aa89b33c75fffc8e1e7) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.899078 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 1ef353f5e715452eae1f1c191fe79d1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9a290b8114844aa89b33c75fffc8e1e7) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.899567 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 1ef353f5e715452eae1f1c191fe79d1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9a290b8114844aa89b33c75fffc8e1e7) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:53.899762 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1ef353f5e715452eae1f1c191fe79d1d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ef353f5e715452eae1f1c191fe79d1d already in progress: creating tablet
I20251028 02:59:53.900820 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_e4f1369ad6ad4110b5854544d4a266bd [id=ccea14281b2c420a99cede953a8de44d] alter complete (version 1)
I20251028 02:59:53.900897 9011 raft_consensus.cc:493] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.900979 9011 raft_consensus.cc:515] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.901374 9011 leader_election.cc:290] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.901867 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c048520fe7b248e586e44a2047ffb239" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:53.901978 8420 raft_consensus.cc:2468] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:53.902043 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ae2a510d64243a9b0fb7784ee56d29f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.902216 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ae2a510d64243a9b0fb7784ee56d29f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.902452 8220 leader_election.cc:304] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.902475 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet da1563d65b4d4ab48325a4f08ee89ed5 from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:53.902683 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 8ae2a510d64243a9b0fb7784ee56d29f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8ae2a510d64243a9b0fb7784ee56d29f already in progress: creating tablet
I20251028 02:59:53.902730 8147 raft_consensus.cc:3060] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.903864 9011 raft_consensus.cc:2804] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.903928 9011 raft_consensus.cc:493] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.903976 9011 raft_consensus.cc:3060] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.904140 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c048520fe7b248e586e44a2047ffb239" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.904242 8144 raft_consensus.cc:2468] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:53.895112 8551 ts_tablet_manager.cc:1929] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.904443 8551 log.cc:1199] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/e4cdbf69eceb4bec9de3368ec959d5f0
I20251028 02:59:53.904713 8965 consensus_queue.cc:1048] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.905012 8551 ts_tablet_manager.cc:1950] T e4cdbf69eceb4bec9de3368ec959d5f0 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.905063 9011 raft_consensus.cc:515] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.905413 9011 leader_election.cc:290] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.905853 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c048520fe7b248e586e44a2047ffb239" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:53.905964 8420 raft_consensus.cc:3060] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.906639 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet e4cdbf69eceb4bec9de3368ec959d5f0 (table default.Table_50c26dd07fa04d9a838968de2a92f0e5 [id=5290176e2166464799b4d2f4d54e97ff]) successfully deleted
I20251028 02:59:53.906911 8420 raft_consensus.cc:2468] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
W20251028 02:59:53.907086 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 1ef353f5e715452eae1f1c191fe79d1d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ef353f5e715452eae1f1c191fe79d1d already in progress: creating tablet
I20251028 02:59:53.907269 8220 leader_election.cc:304] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.908145 9011 raft_consensus.cc:2804] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:53.908236 9011 raft_consensus.cc:697] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:53.908366 9011 consensus_queue.cc:237] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.908597 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c048520fe7b248e586e44a2047ffb239" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:53.908700 8147 raft_consensus.cc:3060] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.909653 8147 raft_consensus.cc:2468] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:53.910962 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet da1563d65b4d4ab48325a4f08ee89ed5
I20251028 02:59:53.911939 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ae2a510d64243a9b0fb7784ee56d29f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.912364 8551 tablet_replica.cc:333] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.912448 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet da1563d65b4d4ab48325a4f08ee89ed5 with delete_type TABLET_DATA_DELETED (replaced by tablet 529b865389f94471a6849d17ff206e4c) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.912595 8550 tablet_replica.cc:333] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:53.912611 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet da1563d65b4d4ab48325a4f08ee89ed5 with delete_type TABLET_DATA_DELETED (replaced by tablet 529b865389f94471a6849d17ff206e4c) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.912712 8550 raft_consensus.cc:2243] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.912817 8550 raft_consensus.cc:2272] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.912922 7962 catalog_manager.cc:6719] ReplaceTablet: tablet da1563d65b4d4ab48325a4f08ee89ed5 deleted and replaced by tablet 529b865389f94471a6849d17ff206e4c
I20251028 02:59:53.912521 8551 raft_consensus.cc:2243] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.913208 8551 raft_consensus.cc:2272] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.912377 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet da1563d65b4d4ab48325a4f08ee89ed5 with delete_type TABLET_DATA_DELETED (replaced by tablet 529b865389f94471a6849d17ff206e4c) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.913610 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet 7528e7ff127c475789eda805467f0b41 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:53.914073 8551 ts_tablet_manager.cc:1916] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.915660 8550 ts_tablet_manager.cc:1916] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.917158 8551 ts_tablet_manager.cc:1929] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.917292 8551 log.cc:1199] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8ae2a510d64243a9b0fb7784ee56d29f
I20251028 02:59:53.917551 8551 ts_tablet_manager.cc:1950] T 8ae2a510d64243a9b0fb7784ee56d29f P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.917695 8965 consensus_queue.cc:1048] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.917999 8550 ts_tablet_manager.cc:1929] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.918078 8550 log.cc:1199] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/da1563d65b4d4ab48325a4f08ee89ed5
I20251028 02:59:53.918287 8550 ts_tablet_manager.cc:1950] T da1563d65b4d4ab48325a4f08ee89ed5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:53.918478 8551 tablet_replica.cc:333] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.918654 8551 raft_consensus.cc:2243] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.918800 8551 raft_consensus.cc:2272] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.919212 8551 ts_tablet_manager.cc:1916] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.920578 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8ae2a510d64243a9b0fb7784ee56d29f (table default.Table_23ea93791ab84db8acf0b4e6c21141d8 [id=2fccc00ed79649eabd08b00696a27710]) successfully deleted
I20251028 02:59:53.921890 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet da1563d65b4d4ab48325a4f08ee89ed5 (table default.Table_153d49d056a3458a9565c70d954187ae [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
I20251028 02:59:53.922897 8551 ts_tablet_manager.cc:1929] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.924031 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 7528e7ff127c475789eda805467f0b41
I20251028 02:59:53.924181 8551 log.cc:1199] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/da1563d65b4d4ab48325a4f08ee89ed5
I20251028 02:59:53.924502 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 7528e7ff127c475789eda805467f0b41 with delete_type TABLET_DATA_DELETED (replaced by tablet 5428db004de74f7492c9b8999e53020e) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.924636 7962 catalog_manager.cc:6719] ReplaceTablet: tablet 7528e7ff127c475789eda805467f0b41 deleted and replaced by tablet 5428db004de74f7492c9b8999e53020e
I20251028 02:59:53.924746 8551 ts_tablet_manager.cc:1950] T da1563d65b4d4ab48325a4f08ee89ed5 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.925201 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 7528e7ff127c475789eda805467f0b41 with delete_type TABLET_DATA_DELETED (replaced by tablet 5428db004de74f7492c9b8999e53020e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.925698 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet da1563d65b4d4ab48325a4f08ee89ed5 (table default.Table_153d49d056a3458a9565c70d954187ae [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
W20251028 02:59:53.925706 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 7528e7ff127c475789eda805467f0b41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7528e7ff127c475789eda805467f0b41 already in progress: creating tablet
I20251028 02:59:53.927531 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 7528e7ff127c475789eda805467f0b41 with delete_type TABLET_DATA_DELETED (replaced by tablet 5428db004de74f7492c9b8999e53020e) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:53.928045 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 7528e7ff127c475789eda805467f0b41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7528e7ff127c475789eda805467f0b41 already in progress: creating tablet
I20251028 02:59:53.928123 8546 tablet_bootstrap.cc:492] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.928220 8546 ts_tablet_manager.cc:1403] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20251028 02:59:53.928732 8546 raft_consensus.cc:359] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.928869 8546 raft_consensus.cc:385] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.928905 8546 raft_consensus.cc:740] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.928956 7786 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
/tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.189:45473
--webserver_interface=127.7.154.189
--webserver_port=36469
--builtin_ntp_servers=127.7.154.148:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20251028 02:59:53.929181 8546 consensus_queue.cc:260] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.932219 8546 ts_tablet_manager.cc:1434] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:53.932451 8546 tablet_bootstrap.cc:492] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.933219 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0424d53c61864c5cbaeb7c12b0b6d96c" } new_table_name: "default.Table_fa08b2c03e264b18ba4f30628531ed79" modify_external_catalogs: true
I20251028 02:59:53.933389 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_8b7624342986474fbd0e31c86947f781"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.933745 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b7624342986474fbd0e31c86947f781 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.934005 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1d2bfae88392496383bd49d6f3dd832f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.934355 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1d2bfae88392496383bd49d6f3dd832f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.935148 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3292bf8a27264e8491009af851138333"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:53.935447 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3292bf8a27264e8491009af851138333 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:53.936256 8546 tablet_bootstrap.cc:654] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.936319 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b883c2cd0a0f49fca261f85e62e9bc18" } new_table_name: "default.Table_663247205d844151bccc25218dcf39a5" modify_external_catalogs: true
I20251028 02:59:53.937534 8552 ts_tablet_manager.cc:1929] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.937616 8552 log.cc:1199] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/aec9427ed4284d66b9bd46c0e1c8774c
I20251028 02:59:53.937644 8546 tablet_bootstrap.cc:492] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.937741 8546 ts_tablet_manager.cc:1403] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:53.937867 8552 ts_tablet_manager.cc:1950] T aec9427ed4284d66b9bd46c0e1c8774c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.938228 8546 raft_consensus.cc:359] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.938347 8546 raft_consensus.cc:385] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.938391 8546 raft_consensus.cc:740] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.938519 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet aec9427ed4284d66b9bd46c0e1c8774c (table default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]) successfully deleted
I20251028 02:59:53.938555 8546 consensus_queue.cc:260] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.938890 8546 ts_tablet_manager.cc:1434] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.939100 8546 tablet_bootstrap.cc:492] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.941699 8546 tablet_bootstrap.cc:654] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.941823 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 06302d2cb5554805b85916741ea10f0c (DEFAULT_TABLE table=default.Table_88e852b7e3584db7bfcfd5175978ade6 [id=ef18ab5f49634ad3b7174f0083b4b71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.942134 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06302d2cb5554805b85916741ea10f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.943408 8544 tablet_bootstrap.cc:492] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.943513 8544 ts_tablet_manager.cc:1403] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.075s user 0.002s sys 0.000s
I20251028 02:59:53.943549 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 06302d2cb5554805b85916741ea10f0c (DEFAULT_TABLE table=default.Table_88e852b7e3584db7bfcfd5175978ade6 [id=ef18ab5f49634ad3b7174f0083b4b71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.943825 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06302d2cb5554805b85916741ea10f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.943969 8546 tablet_bootstrap.cc:492] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.944047 8546 ts_tablet_manager.cc:1403] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:53.943993 8544 raft_consensus.cc:359] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.944094 8544 raft_consensus.cc:385] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.944133 8544 raft_consensus.cc:740] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.944253 8544 consensus_queue.cc:260] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.944494 8544 ts_tablet_manager.cc:1434] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.944522 8546 raft_consensus.cc:359] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.944624 8546 raft_consensus.cc:385] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.944656 8546 raft_consensus.cc:740] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.944689 8544 tablet_bootstrap.cc:492] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.944766 8546 consensus_queue.cc:260] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.945078 8546 ts_tablet_manager.cc:1434] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.945259 8546 tablet_bootstrap.cc:492] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.945654 8544 tablet_bootstrap.cc:654] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.946040 8394 tablet_service.cc:1505] Processing CreateTablet for tablet fef84b69b1b746d4b4feeed96670c67b (DEFAULT_TABLE table=default.Table_2fd19ddb9673469a98456a4520898183 [id=cf715387170e40c992807874e34c66b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.946237 8546 tablet_bootstrap.cc:654] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.946282 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fef84b69b1b746d4b4feeed96670c67b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.947394 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 49b0b3459d434cf2810029ce205e745c (DEFAULT_TABLE table=default.Table_75e654d01f4147b3b7d928c99967c5cc [id=f9132fe51a804151b8df8e1e8fc0d220]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.947504 9011 raft_consensus.cc:493] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.947573 9011 raft_consensus.cc:515] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.947634 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b0b3459d434cf2810029ce205e745c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.947926 9011 leader_election.cc:290] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:53.948599 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 06302d2cb5554805b85916741ea10f0c (DEFAULT_TABLE table=default.Table_88e852b7e3584db7bfcfd5175978ade6 [id=ef18ab5f49634ad3b7174f0083b4b71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.948815 8546 tablet_bootstrap.cc:492] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.948841 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06302d2cb5554805b85916741ea10f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.948889 8546 ts_tablet_manager.cc:1403] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:53.949330 8546 raft_consensus.cc:359] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:53.949373 8220 leader_election.cc:343] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: da1563d65b4d4ab48325a4f08ee89ed5
I20251028 02:59:53.949415 8546 raft_consensus.cc:385] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.949445 8546 raft_consensus.cc:740] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.949550 8546 consensus_queue.cc:260] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.949880 8546 ts_tablet_manager.cc:1434] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.950058 8546 tablet_bootstrap.cc:492] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.951161 8546 tablet_bootstrap.cc:654] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.951133 8258 tablet_service.cc:1505] Processing CreateTablet for tablet fef84b69b1b746d4b4feeed96670c67b (DEFAULT_TABLE table=default.Table_2fd19ddb9673469a98456a4520898183 [id=cf715387170e40c992807874e34c66b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.951592 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fef84b69b1b746d4b4feeed96670c67b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.951851 8552 tablet_replica.cc:333] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:53.951982 8552 raft_consensus.cc:2243] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.952199 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 37dcf605a2984be1817d9621e4a41448 (table default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc]). Expected version 1 got 0
W20251028 02:59:53.952207 8218 leader_election.cc:343] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: da1563d65b4d4ab48325a4f08ee89ed5
I20251028 02:59:53.952270 8218 leader_election.cc:304] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:53.952371 8965 raft_consensus.cc:2749] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:53.952082 8552 raft_consensus.cc:2272] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.954780 8118 tablet_service.cc:1505] Processing CreateTablet for tablet fef84b69b1b746d4b4feeed96670c67b (DEFAULT_TABLE table=default.Table_2fd19ddb9673469a98456a4520898183 [id=cf715387170e40c992807874e34c66b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.955099 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fef84b69b1b746d4b4feeed96670c67b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.957262 8552 ts_tablet_manager.cc:1916] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:53.957505 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet e4cdbf69eceb4bec9de3368ec959d5f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e4cdbf69eceb4bec9de3368ec959d5f0 already in progress: creating tablet
I20251028 02:59:53.957592 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 12dce4bb3d4b43608d7da9350b14e1d0
W20251028 02:59:53.957681 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet eaa41f8bf7c54673b65893acc59052c1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet eaa41f8bf7c54673b65893acc59052c1 already in progress: creating tablet
I20251028 02:59:53.957815 7968 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 68ca4ff164b24b7498f953ec11cf46be (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:53.957844 7967 catalog_manager.cc:6719] ReplaceTablet: tablet 12dce4bb3d4b43608d7da9350b14e1d0 deleted and replaced by tablet a808098c70b54ad3a0230352af0b743c
I20251028 02:59:53.958137 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 03959a61996b40d8b9ef8b4b9708f017
I20251028 02:59:53.958909 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 12dce4bb3d4b43608d7da9350b14e1d0 with delete_type TABLET_DATA_DELETED (replaced by tablet a808098c70b54ad3a0230352af0b743c) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.959618 8552 ts_tablet_manager.cc:1929] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.959689 8552 log.cc:1199] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/02f6a32db68e4c609da39c3ee6c97348
I20251028 02:59:53.959702 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 12dce4bb3d4b43608d7da9350b14e1d0 with delete_type TABLET_DATA_DELETED (replaced by tablet a808098c70b54ad3a0230352af0b743c) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.959918 8552 ts_tablet_manager.cc:1950] T 02f6a32db68e4c609da39c3ee6c97348 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:53.960110 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 03959a61996b40d8b9ef8b4b9708f017 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:53.960281 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 12dce4bb3d4b43608d7da9350b14e1d0 with delete_type TABLET_DATA_DELETED (replaced by tablet a808098c70b54ad3a0230352af0b743c) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:53.960387 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 03959a61996b40d8b9ef8b4b9708f017 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:53.960630 8551 tablet_replica.cc:333] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:53.960678 8552 tablet_replica.cc:333] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 02:59:53.960706 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 12dce4bb3d4b43608d7da9350b14e1d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12dce4bb3d4b43608d7da9350b14e1d0 already in progress: creating tablet
I20251028 02:59:53.960778 8551 raft_consensus.cc:2243] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.960827 8552 raft_consensus.cc:2243] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:53.960922 8552 raft_consensus.cc:2272] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.960914 8551 raft_consensus.cc:2272] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:53.961445 8551 ts_tablet_manager.cc:1916] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:53.961457 8545 tablet_bootstrap.cc:492] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.961577 8545 ts_tablet_manager.cc:1403] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.097s user 0.001s sys 0.001s
I20251028 02:59:53.961819 8420 raft_consensus.cc:1275] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:53.962049 8545 raft_consensus.cc:359] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.962139 8545 raft_consensus.cc:385] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.962167 8545 raft_consensus.cc:740] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.962225 9011 consensus_queue.cc:1048] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.962282 8545 consensus_queue.cc:260] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.962551 8545 ts_tablet_manager.cc:1434] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.962759 8545 tablet_bootstrap.cc:492] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.963726 8551 ts_tablet_manager.cc:1929] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:53.963793 8551 log.cc:1199] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/03959a61996b40d8b9ef8b4b9708f017
I20251028 02:59:53.964006 8551 ts_tablet_manager.cc:1950] T 03959a61996b40d8b9ef8b4b9708f017 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:53.964082 7968 catalog_manager.cc:5649] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:53.964303 7968 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]). Expected version 1 got 0
I20251028 02:59:53.964418 7968 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_0e8ca626159c41aeb8e360f6e038f7a3 [id=eb0e103661fc4bc7b27bb78fc333fd56]). Expected version 1 got 0
I20251028 02:59:53.964663 8147 raft_consensus.cc:3060] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.965080 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]). Expected version 1 got 0
I20251028 02:59:53.965703 8147 raft_consensus.cc:1275] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:53.967056 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 13de247dfb18478daac9ca5601220faa (DEFAULT_TABLE table=default.Table_f4eb60b8380645758ce2cadfaff1330a [id=e5dbece40326467088d7435702d8acbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.967222 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 4e44345b09584bbca19b19e57ce17c77 (DEFAULT_TABLE table=default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.967247 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 49b46c13cb4e46388ed0a69d2ced9b42 (DEFAULT_TABLE table=default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.967242 8556 consensus_queue.cc:1048] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:53.967358 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13de247dfb18478daac9ca5601220faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.967497 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b46c13cb4e46388ed0a69d2ced9b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.967525 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 04d271c95c2847ca8b1fd2d55741117c (DEFAULT_TABLE table=default.Table_ad3466e031ae40059b8e3ec01c7ed5a9 [id=f00f5e8a1fef48e480906cbf79121e33]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.967700 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04d271c95c2847ca8b1fd2d55741117c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.969056 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]). Expected version 1 got 0
I20251028 02:59:53.969159 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet df10bbbd757a4da8905718b61cbb10c8 (table default.Table_bb39271bc8354db88ef788080eb177bc [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 3 got 0
I20251028 02:59:53.969663 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 74120decd53d4de19c5f6fedf70ecaae (DEFAULT_TABLE table=default.Table_50c26dd07fa04d9a838968de2a92f0e5 [id=5290176e2166464799b4d2f4d54e97ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.969784 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e44345b09584bbca19b19e57ce17c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.970250 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5f8a6ff79e8f4ad7aee8aa99b07f20ba" } modify_external_catalogs: true
I20251028 02:59:53.970361 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5f8a6ff79e8f4ad7aee8aa99b07f20ba" } modify_external_catalogs: true
I20251028 02:59:53.972767 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 03959a61996b40d8b9ef8b4b9708f017 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:53.973665 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5428db004de74f7492c9b8999e53020e has no consensus state
I20251028 02:59:53.976356 8545 tablet_bootstrap.cc:654] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.977205 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 49b0b3459d434cf2810029ce205e745c (DEFAULT_TABLE table=default.Table_75e654d01f4147b3b7d928c99967c5cc [id=f9132fe51a804151b8df8e1e8fc0d220]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.977357 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 74120decd53d4de19c5f6fedf70ecaae (DEFAULT_TABLE table=default.Table_50c26dd07fa04d9a838968de2a92f0e5 [id=5290176e2166464799b4d2f4d54e97ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.977474 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b0b3459d434cf2810029ce205e745c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.977603 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74120decd53d4de19c5f6fedf70ecaae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.979140 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74120decd53d4de19c5f6fedf70ecaae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.980100 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 13de247dfb18478daac9ca5601220faa (DEFAULT_TABLE table=default.Table_f4eb60b8380645758ce2cadfaff1330a [id=e5dbece40326467088d7435702d8acbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.980329 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 4e44345b09584bbca19b19e57ce17c77 (DEFAULT_TABLE table=default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.980378 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13de247dfb18478daac9ca5601220faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.980484 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 49b0b3459d434cf2810029ce205e745c (DEFAULT_TABLE table=default.Table_75e654d01f4147b3b7d928c99967c5cc [id=f9132fe51a804151b8df8e1e8fc0d220]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.980543 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e44345b09584bbca19b19e57ce17c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.980692 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b0b3459d434cf2810029ce205e745c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.980100 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 74120decd53d4de19c5f6fedf70ecaae (DEFAULT_TABLE table=default.Table_50c26dd07fa04d9a838968de2a92f0e5 [id=5290176e2166464799b4d2f4d54e97ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.982545 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74120decd53d4de19c5f6fedf70ecaae. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:53.982811 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 03959a61996b40d8b9ef8b4b9708f017 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 03959a61996b40d8b9ef8b4b9708f017 already in progress: creating tablet
I20251028 02:59:53.983031 8544 tablet_bootstrap.cc:492] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.983124 8544 ts_tablet_manager.cc:1403] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20251028 02:59:53.983440 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 13de247dfb18478daac9ca5601220faa (DEFAULT_TABLE table=default.Table_f4eb60b8380645758ce2cadfaff1330a [id=e5dbece40326467088d7435702d8acbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.983669 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13de247dfb18478daac9ca5601220faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.983645 8544 raft_consensus.cc:359] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.983745 8544 raft_consensus.cc:385] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.983785 8544 raft_consensus.cc:740] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.983902 8544 consensus_queue.cc:260] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.984172 8544 ts_tablet_manager.cc:1434] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:53.984350 8544 tablet_bootstrap.cc:492] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.985508 8544 tablet_bootstrap.cc:654] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.986009 8938 raft_consensus.cc:493] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:53.986109 8938 raft_consensus.cc:515] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.986462 8938 leader_election.cc:290] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.987200 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 4e44345b09584bbca19b19e57ce17c77 (DEFAULT_TABLE table=default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.987280 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7f804f81d70749b5bdefeca4e0bad6fe" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:53.987356 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 04d271c95c2847ca8b1fd2d55741117c (DEFAULT_TABLE table=default.Table_ad3466e031ae40059b8e3ec01c7ed5a9 [id=f00f5e8a1fef48e480906cbf79121e33]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.987423 8283 raft_consensus.cc:2468] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:53.987498 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e44345b09584bbca19b19e57ce17c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.987200 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 49b46c13cb4e46388ed0a69d2ced9b42 (DEFAULT_TABLE table=default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.987700 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b46c13cb4e46388ed0a69d2ced9b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.988198 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 04d271c95c2847ca8b1fd2d55741117c (DEFAULT_TABLE table=default.Table_ad3466e031ae40059b8e3ec01c7ed5a9 [id=f00f5e8a1fef48e480906cbf79121e33]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.988440 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04d271c95c2847ca8b1fd2d55741117c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.989542 8545 tablet_bootstrap.cc:492] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:53.989626 8545 ts_tablet_manager.cc:1403] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20251028 02:59:53.989836 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04d271c95c2847ca8b1fd2d55741117c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.989815 8356 leader_election.cc:304] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:53.990402 8616 raft_consensus.cc:2804] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:53.990453 8616 raft_consensus.cc:493] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:53.990489 8616 raft_consensus.cc:3060] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:53.990674 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7f804f81d70749b5bdefeca4e0bad6fe" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:53.991271 8544 tablet_bootstrap.cc:492] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.991418 8544 ts_tablet_manager.cc:1403] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:53.991428 9048 tablet.cc:1722] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:53.991622 8616 raft_consensus.cc:515] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.991916 8544 raft_consensus.cc:359] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.991997 8616 leader_election.cc:290] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:53.992054 8544 raft_consensus.cc:385] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.992097 8544 raft_consensus.cc:740] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.992228 8544 consensus_queue.cc:260] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:53.992447 8545 raft_consensus.cc:359] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.992561 8545 raft_consensus.cc:385] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.992604 8545 raft_consensus.cc:740] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.992578 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 49b46c13cb4e46388ed0a69d2ced9b42 (DEFAULT_TABLE table=default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:53.992801 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b46c13cb4e46388ed0a69d2ced9b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:53.992748 8545 consensus_queue.cc:260] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.993144 8545 ts_tablet_manager.cc:1434] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:53.993371 8545 tablet_bootstrap.cc:492] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:53.994071 8544 ts_tablet_manager.cc:1434] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:53.994692 8545 tablet_bootstrap.cc:654] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.994855 8544 tablet_bootstrap.cc:492] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:53.996770 8544 tablet_bootstrap.cc:654] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:53.997188 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7f804f81d70749b5bdefeca4e0bad6fe" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:53.997244 8354 leader_election.cc:343] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:53.997409 9023 tablet.cc:1722] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:53.997726 8546 tablet_bootstrap.cc:492] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:53.997845 8546 ts_tablet_manager.cc:1403] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.048s user 0.000s sys 0.002s
I20251028 02:59:53.998323 8546 raft_consensus.cc:359] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.998448 8546 raft_consensus.cc:385] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:53.998485 8546 raft_consensus.cc:740] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:53.998474 9122 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:53.998612 8546 consensus_queue.cc:260] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:53.998904 8546 ts_tablet_manager.cc:1434] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:53.999079 8546 tablet_bootstrap.cc:492] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:53.999918 8544 tablet_bootstrap.cc:492] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:53.999998 8544 ts_tablet_manager.cc:1403] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20251028 02:59:54.000164 8546 tablet_bootstrap.cc:654] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.000479 8544 raft_consensus.cc:359] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.000590 8544 raft_consensus.cc:385] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.000624 8544 raft_consensus.cc:740] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.000746 8544 consensus_queue.cc:260] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.001005 8544 ts_tablet_manager.cc:1434] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:54.001179 8544 tablet_bootstrap.cc:492] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.002761 8546 tablet_bootstrap.cc:492] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.002851 8546 ts_tablet_manager.cc:1403] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:54.003206 8544 tablet_bootstrap.cc:654] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.003330 8546 raft_consensus.cc:359] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.003432 8546 raft_consensus.cc:385] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.003464 8546 raft_consensus.cc:740] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.003584 8546 consensus_queue.cc:260] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.003880 8546 ts_tablet_manager.cc:1434] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.004053 8546 tablet_bootstrap.cc:492] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:54.004068 8354 leader_election.cc:343] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.004638 9023 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:54.004995 8546 tablet_bootstrap.cc:654] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.006958 8546 tablet_bootstrap.cc:492] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.007052 8546 ts_tablet_manager.cc:1403] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:54.008162 8546 raft_consensus.cc:359] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.008193 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7f804f81d70749b5bdefeca4e0bad6fe" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:54.008270 8546 raft_consensus.cc:385] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.008306 8283 raft_consensus.cc:3060] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.008304 8546 raft_consensus.cc:740] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.008494 8546 consensus_queue.cc:260] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.008957 9122 tablet.cc:1722] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:54.009356 8283 raft_consensus.cc:2468] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:54.009390 8546 ts_tablet_manager.cc:1434] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251028 02:59:54.010090 8546 tablet_bootstrap.cc:492] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.011117 8546 tablet_bootstrap.cc:654] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.012145 8552 ts_tablet_manager.cc:1916] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.012832 9048 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 02:59:54.014317 8552 ts_tablet_manager.cc:1929] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.014382 8552 log.cc:1199] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/8ae2a510d64243a9b0fb7784ee56d29f
I20251028 02:59:54.015844 8356 leader_election.cc:304] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.016196 8616 raft_consensus.cc:2804] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:54.016278 8616 raft_consensus.cc:697] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:54.016399 8616 consensus_queue.cc:237] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.018136 8552 ts_tablet_manager.cc:1950] T 8ae2a510d64243a9b0fb7784ee56d29f P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.018328 8965 consensus_queue.cc:1048] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.020805 7794 master_runner.cc:186] Error getting master registration for 127.7.154.189:45473: Network error: Client connection negotiation failed: client connection to 127.7.154.189:45473: connect: Connection refused (error 111),
I20251028 02:59:54.023155 7794 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251028 02:59:54.024041 8552 tablet_replica.cc:333] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.024315 8552 raft_consensus.cc:2243] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.024519 8552 raft_consensus.cc:2272] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.025199 8552 ts_tablet_manager.cc:1916] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.025457 8544 tablet_bootstrap.cc:492] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.025549 8544 ts_tablet_manager.cc:1403] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.001s
I20251028 02:59:54.025956 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet df10bbbd757a4da8905718b61cbb10c8 (table default.Table_bb39271bc8354db88ef788080eb177bc [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 3 got 0
I20251028 02:59:54.026245 8544 raft_consensus.cc:359] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.026353 8544 raft_consensus.cc:385] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.026388 8544 raft_consensus.cc:740] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.026516 8544 consensus_queue.cc:260] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.026815 8544 ts_tablet_manager.cc:1434] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.026986 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet df10bbbd757a4da8905718b61cbb10c8 (table default.Table_bb39271bc8354db88ef788080eb177bc [id=a6ae2be5375e4328848f28f43dfc49ab]). Expected version 3 got 0
I20251028 02:59:54.027034 8544 tablet_bootstrap.cc:492] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.028056 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 002de8aede994720b8163f41938af6c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.028307 8545 tablet_bootstrap.cc:492] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.028405 8545 ts_tablet_manager.cc:1403] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.002s
I20251028 02:59:54.028491 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c87d03fad194ca89f1808da6dbdf0a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a3096fde2ac94ab197ce5e659c4d2be0) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.028950 8545 raft_consensus.cc:359] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.029062 8545 raft_consensus.cc:385] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.029099 8545 raft_consensus.cc:740] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.029251 8545 consensus_queue.cc:260] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.029990 8549 raft_consensus.cc:493] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.030078 8549 raft_consensus.cc:515] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.030323 8545 ts_tablet_manager.cc:1434] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 02:59:54.030422 8549 leader_election.cc:290] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.030509 8545 tablet_bootstrap.cc:492] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.031832 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50e6d41d1cf746aab2802b5c6b2d58b3" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:54.031903 8552 ts_tablet_manager.cc:1929] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.031976 8552 log.cc:1199] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/da1563d65b4d4ab48325a4f08ee89ed5
I20251028 02:59:54.032224 8552 ts_tablet_manager.cc:1950] T da1563d65b4d4ab48325a4f08ee89ed5 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 02:59:54.032596 8354 leader_election.cc:343] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.033053 8552 tablet_replica.cc:333] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.033197 8552 raft_consensus.cc:2243] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.033309 8552 raft_consensus.cc:2272] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.033581 8545 tablet_bootstrap.cc:654] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.034188 8544 tablet_bootstrap.cc:654] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.034315 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ec69044a4c3e4f049f90be835cf73367
I20251028 02:59:54.035336 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet ec69044a4c3e4f049f90be835cf73367 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.035620 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50e6d41d1cf746aab2802b5c6b2d58b3" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.036108 8552 ts_tablet_manager.cc:1916] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:54.035988 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ec69044a4c3e4f049f90be835cf73367 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec69044a4c3e4f049f90be835cf73367 already in progress: creating tablet
W20251028 02:59:54.036463 8356 leader_election.cc:343] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.036629 8356 leader_election.cc:304] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.036643 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 9c1d8552cc1941bda2e9f02fba56c9f9 (table default.Table_bb551b322c5140ba89d154ff450085dd [id=4e8c5072da4e4ec6a616ba4c7fc91cd3]). Expected version 1 got 0
I20251028 02:59:54.036900 8549 raft_consensus.cc:2749] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.039508 8552 ts_tablet_manager.cc:1929] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.039590 8552 log.cc:1199] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/03959a61996b40d8b9ef8b4b9708f017
I20251028 02:59:54.039829 8552 ts_tablet_manager.cc:1950] T 03959a61996b40d8b9ef8b4b9708f017 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.040957 8552 tablet_replica.cc:333] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.041100 8552 raft_consensus.cc:2243] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.041194 8552 raft_consensus.cc:2272] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.042004 8552 ts_tablet_manager.cc:1916] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.042543 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 02f6a32db68e4c609da39c3ee6c97348 (table default.Table_e69b7cea080549b1b8a67cb91703a472 [id=0c94092ef3844117af5e2aa5086dabb6]) successfully deleted
W20251028 02:59:54.042732 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 9feee73521df42ae89f5af5c5eacf9bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9feee73521df42ae89f5af5c5eacf9bd already in progress: creating tablet
W20251028 02:59:54.042853 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 1ef353f5e715452eae1f1c191fe79d1d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ef353f5e715452eae1f1c191fe79d1d already in progress: creating tablet
I20251028 02:59:54.043251 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 8ae2a510d64243a9b0fb7784ee56d29f (table default.Table_23ea93791ab84db8acf0b4e6c21141d8 [id=2fccc00ed79649eabd08b00696a27710]) successfully deleted
I20251028 02:59:54.043390 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet da1563d65b4d4ab48325a4f08ee89ed5 (table default.Table_153d49d056a3458a9565c70d954187ae [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
W20251028 02:59:54.043718 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 7528e7ff127c475789eda805467f0b41 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7528e7ff127c475789eda805467f0b41 already in progress: creating tablet
W20251028 02:59:54.043848 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 12dce4bb3d4b43608d7da9350b14e1d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12dce4bb3d4b43608d7da9350b14e1d0 already in progress: creating tablet
I20251028 02:59:54.043938 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 03959a61996b40d8b9ef8b4b9708f017 (table default.Table_5da94c10f01846a59e4ce4ff530330d3 [id=bc83d2963bb74c2d9b914dee53bcf6ba]) successfully deleted
W20251028 02:59:54.044018 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 12dce4bb3d4b43608d7da9350b14e1d0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12dce4bb3d4b43608d7da9350b14e1d0 already in progress: creating tablet
I20251028 02:59:54.044027 8544 tablet_bootstrap.cc:492] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.044090 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 03959a61996b40d8b9ef8b4b9708f017 (table default.Table_5da94c10f01846a59e4ce4ff530330d3 [id=bc83d2963bb74c2d9b914dee53bcf6ba]) successfully deleted
I20251028 02:59:54.044127 8544 ts_tablet_manager.cc:1403] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 02:59:54.044601 8544 raft_consensus.cc:359] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.044656 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet ec69044a4c3e4f049f90be835cf73367 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.044715 8544 raft_consensus.cc:385] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.044746 8544 raft_consensus.cc:740] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.044869 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet ec69044a4c3e4f049f90be835cf73367 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.044893 8544 consensus_queue.cc:260] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.045188 8544 ts_tablet_manager.cc:1434] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.045390 8544 tablet_bootstrap.cc:492] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.045404 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet cf90b539dc8f47b0947d4e2ca859668a from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.045534 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c510beb7614e484c8b45ff008bca293c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.045786 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c7e0f5beff234f17b69504efc678924d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.045976 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c510beb7614e484c8b45ff008bca293c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:54.046047 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ec69044a4c3e4f049f90be835cf73367 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec69044a4c3e4f049f90be835cf73367 already in progress: creating tablet
W20251028 02:59:54.046123 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c7e0f5beff234f17b69504efc678924d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.046506 8544 tablet_bootstrap.cc:654] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.046749 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bb39271bc8354db88ef788080eb177bc" } new_table_name: "default.Table_05d02074a9864884801e6bbd08265546" modify_external_catalogs: true
I20251028 02:59:54.046880 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2de1f8c1f3564959a40963351d7d60b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.046962 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bb551b322c5140ba89d154ff450085dd" } modify_external_catalogs: true
I20251028 02:59:54.047045 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bb551b322c5140ba89d154ff450085dd" } modify_external_catalogs: true
W20251028 02:59:54.047186 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2de1f8c1f3564959a40963351d7d60b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.047818 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bd14742bd9f943cc93df997dfecd2719"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.048101 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd14742bd9f943cc93df997dfecd2719 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.048148 8420 raft_consensus.cc:1275] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:54.048554 8556 consensus_queue.cc:1048] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.048799 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_12396134528e4544a80934aa91d07353" } new_table_name: "default.Table_f9254be0618c49f8a6a36fb9fc10354a" modify_external_catalogs: true
I20251028 02:59:54.049557 8144 raft_consensus.cc:3060] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.050685 8144 raft_consensus.cc:1275] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:54.051416 8556 consensus_queue.cc:1048] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.052896 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_610f127f3f9348309aa1ce4b6537e5b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.053267 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_610f127f3f9348309aa1ce4b6537e5b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.055413 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bd4137d37a9847a38ea0d06c10c70186"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.055712 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd4137d37a9847a38ea0d06c10c70186 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.055846 8552 ts_tablet_manager.cc:1929] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.055919 8552 log.cc:1199] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/002de8aede994720b8163f41938af6c4
I20251028 02:59:54.056120 8552 ts_tablet_manager.cc:1950] T 002de8aede994720b8163f41938af6c4 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.056743 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 002de8aede994720b8163f41938af6c4 (table default.Table_c3d6504130c54bdeade5bb23b6cd0679 [id=371c4929d59942f79f41ad071b61e017]) successfully deleted
I20251028 02:59:54.056813 8552 tablet_replica.cc:333] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.056949 8552 raft_consensus.cc:2243] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.057061 8552 raft_consensus.cc:2272] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.058255 8544 tablet_bootstrap.cc:492] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.058364 8544 ts_tablet_manager.cc:1403] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20251028 02:59:54.058858 8544 raft_consensus.cc:359] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.058970 8544 raft_consensus.cc:385] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.059005 8544 raft_consensus.cc:740] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.059130 8544 consensus_queue.cc:260] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.059465 8544 ts_tablet_manager.cc:1434] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.059643 8544 tablet_bootstrap.cc:492] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.060627 8544 tablet_bootstrap.cc:654] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.061101 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_bb39271bc8354db88ef788080eb177bc [id=a6ae2be5375e4328848f28f43dfc49ab] alter complete (version 3)
I20251028 02:59:54.057556 8552 ts_tablet_manager.cc:1916] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.062291 9122 tablet.cc:1722] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.062623 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d611f94943f44256ad3b9ca490f3282c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.062991 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d611f94943f44256ad3b9ca490f3282c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.063225 9048 tablet.cc:1722] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.067929 8562 raft_consensus.cc:493] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.068041 8562 raft_consensus.cc:515] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.068460 8562 leader_election.cc:290] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.069186 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ca4ff164b24b7498f953ec11cf46be" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.069320 8283 raft_consensus.cc:2468] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:54.070326 9023 tablet.cc:1722] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.070803 8083 leader_election.cc:304] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:54.071235 8562 raft_consensus.cc:2804] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:54.071214 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ca4ff164b24b7498f953ec11cf46be" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:54.071286 8562 raft_consensus.cc:493] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:54.071360 8562 raft_consensus.cc:3060] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.071378 8420 raft_consensus.cc:2468] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:54.072352 8562 raft_consensus.cc:515] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.072739 8562 leader_election.cc:290] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.073171 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ca4ff164b24b7498f953ec11cf46be" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:54.073280 8283 raft_consensus.cc:3060] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.073657 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_54bb44e4b3aa4babb80308231391809c" } modify_external_catalogs: true
I20251028 02:59:54.073779 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_54bb44e4b3aa4babb80308231391809c" } modify_external_catalogs: true
I20251028 02:59:54.074203 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9c1d8552cc1941bda2e9f02fba56c9f9
I20251028 02:59:54.074247 8283 raft_consensus.cc:2468] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:54.074602 8083 leader_election.cc:304] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:54.075088 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ca4ff164b24b7498f953ec11cf46be" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:54.075090 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_698073e4869e4f8fbbb8e8f8c548887d" } new_table_name: "default.Table_1306f4d4723b45fbb688bfccebd665a5" modify_external_catalogs: true
I20251028 02:59:54.075192 8420 raft_consensus.cc:3060] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.075631 8546 tablet_bootstrap.cc:492] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.075717 8546 ts_tablet_manager.cc:1403] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.066s user 0.000s sys 0.002s
I20251028 02:59:54.076128 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c1d8552cc1941bda2e9f02fba56c9f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.076210 8420 raft_consensus.cc:2468] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:54.076242 8546 raft_consensus.cc:359] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.076340 8546 raft_consensus.cc:385] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.076371 8546 raft_consensus.cc:740] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.076496 8546 consensus_queue.cc:260] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.076614 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c1d8552cc1941bda2e9f02fba56c9f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.076771 8551 tablet_replica.cc:333] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.076764 8546 ts_tablet_manager.cc:1434] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:54.076896 8551 raft_consensus.cc:2243] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.076969 8546 tablet_bootstrap.cc:492] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.076992 8551 raft_consensus.cc:2272] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.077343 8551 ts_tablet_manager.cc:1916] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.077983 8546 tablet_bootstrap.cc:654] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.078603 8550 tablet_replica.cc:333] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.078733 8550 raft_consensus.cc:2243] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.078835 8550 pending_rounds.cc:70] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:54.078860 8550 pending_rounds.cc:77] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597131978903552 op_type: NO_OP noop_request { }
I20251028 02:59:54.078912 8550 raft_consensus.cc:2889] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:54.078960 8550 raft_consensus.cc:2272] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.080173 8562 raft_consensus.cc:2804] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:54.080262 8562 raft_consensus.cc:697] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:54.080386 8562 consensus_queue.cc:237] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.081008 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 9c1d8552cc1941bda2e9f02fba56c9f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.081370 8552 ts_tablet_manager.cc:1929] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.081440 8552 log.cc:1199] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/6c87d03fad194ca89f1808da6dbdf0a4
I20251028 02:59:54.081660 8552 ts_tablet_manager.cc:1950] T 6c87d03fad194ca89f1808da6dbdf0a4 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.082185 8551 ts_tablet_manager.cc:1929] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.082266 8551 log.cc:1199] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9c1d8552cc1941bda2e9f02fba56c9f9
I20251028 02:59:54.082314 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 6c87d03fad194ca89f1808da6dbdf0a4 (table default.Table_6d8f283a536e46fcbf3d8abcf9d703bb [id=a6d84958fd654087940c9806e2e3fbc0]) successfully deleted
I20251028 02:59:54.082468 8551 ts_tablet_manager.cc:1950] T 9c1d8552cc1941bda2e9f02fba56c9f9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.083365 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9c1d8552cc1941bda2e9f02fba56c9f9 (table default.Table_bb551b322c5140ba89d154ff450085dd [id=4e8c5072da4e4ec6a616ba4c7fc91cd3]) successfully deleted
I20251028 02:59:54.084023 8544 tablet_bootstrap.cc:492] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.084120 8544 ts_tablet_manager.cc:1403] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20251028 02:59:54.084288 9122 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251028 02:59:54.084614 8544 raft_consensus.cc:359] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.084725 8544 raft_consensus.cc:385] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.084762 8544 raft_consensus.cc:740] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.084896 8544 consensus_queue.cc:260] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.085192 8544 ts_tablet_manager.cc:1434] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.085377 8544 tablet_bootstrap.cc:492] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.085879 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_fa08b2c03e264b18ba4f30628531ed79 [id=4d9d3f97f0d541f7a90885b9ab852d83] alter complete (version 2)
I20251028 02:59:54.089357 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 02f6a32db68e4c609da39c3ee6c97348 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.089357 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet a923402bbc734a0cb44aaf590d1e68ff with delete_type TABLET_DATA_DELETED (replaced by tablet 9e260030c0e6423cba37a2e3af1c6400) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.089546 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet aec9427ed4284d66b9bd46c0e1c8774c with delete_type TABLET_DATA_DELETED (replaced by tablet de38b2d54cec4f4e892047f4650f611b) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.089634 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 03959a61996b40d8b9ef8b4b9708f017 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.090092 9023 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20251028 02:59:54.091351 9198 consensus_queue.cc:1175] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20251028 02:59:54.091905 8550 ts_tablet_manager.cc:1916] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:54.093838 8220 consensus_peers.cc:597] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c1d8552cc1941bda2e9f02fba56c9f9. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:54.094139 8550 ts_tablet_manager.cc:1929] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:54.094218 8550 log.cc:1199] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9c1d8552cc1941bda2e9f02fba56c9f9
I20251028 02:59:54.094154 8544 tablet_bootstrap.cc:654] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.094457 8550 ts_tablet_manager.cc:1950] T 9c1d8552cc1941bda2e9f02fba56c9f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 02:59:54.094952 8218 consensus_peers.cc:597] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda -> Peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Couldn't send request to peer fbd2afb4894c4e7a8520cc09f8593aa0. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:54.095034 8550 tablet_replica.cc:333] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.095108 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9c1d8552cc1941bda2e9f02fba56c9f9 (table default.Table_bb551b322c5140ba89d154ff450085dd [id=4e8c5072da4e4ec6a616ba4c7fc91cd3]) successfully deleted
I20251028 02:59:54.095165 8550 raft_consensus.cc:2243] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.095260 8550 raft_consensus.cc:2272] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.095531 9048 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251028 02:59:54.095616 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet e4cdbf69eceb4bec9de3368ec959d5f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 74120decd53d4de19c5f6fedf70ecaae) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.096853 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e9234b72f4448089fe7ff6ba49483b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.097105 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_05d02074a9864884801e6bbd08265546 [id=a6ae2be5375e4328848f28f43dfc49ab] alter complete (version 4)
I20251028 02:59:54.096434 8550 ts_tablet_manager.cc:1916] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.097669 8552 tablet_replica.cc:333] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.097824 8552 raft_consensus.cc:2243] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
W20251028 02:59:54.097527 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ec69044a4c3e4f049f90be835cf73367 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec69044a4c3e4f049f90be835cf73367 already in progress: creating tablet
I20251028 02:59:54.098145 8552 raft_consensus.cc:2272] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.098695 8552 ts_tablet_manager.cc:1916] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.100965 8552 ts_tablet_manager.cc:1929] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:54.101039 8552 log.cc:1199] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9c1d8552cc1941bda2e9f02fba56c9f9
I20251028 02:59:54.101289 8552 ts_tablet_manager.cc:1950] T 9c1d8552cc1941bda2e9f02fba56c9f9 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.101406 9122 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251028 02:59:54.101928 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f73fbea5f42f4caa815469fbb7504f49" } modify_external_catalogs: true
I20251028 02:59:54.102068 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f73fbea5f42f4caa815469fbb7504f49" } modify_external_catalogs: true
I20251028 02:59:54.102368 8544 tablet_bootstrap.cc:492] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.102443 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9c1d8552cc1941bda2e9f02fba56c9f9 (table default.Table_bb551b322c5140ba89d154ff450085dd [id=4e8c5072da4e4ec6a616ba4c7fc91cd3]) successfully deleted
I20251028 02:59:54.102470 8544 ts_tablet_manager.cc:1403] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20251028 02:59:54.102926 8544 raft_consensus.cc:359] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.103044 8544 raft_consensus.cc:385] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.103085 8544 raft_consensus.cc:740] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.103212 8544 consensus_queue.cc:260] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.103521 8544 ts_tablet_manager.cc:1434] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.103714 8544 tablet_bootstrap.cc:492] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.104000 9048 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251028 02:59:54.104480 9023 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20251028 02:59:54.104749 8544 tablet_bootstrap.cc:654] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.106261 8552 tablet_replica.cc:333] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.106868 8552 raft_consensus.cc:2243] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.106988 8552 raft_consensus.cc:2272] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.108340 8552 ts_tablet_manager.cc:1916] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.109913 8550 ts_tablet_manager.cc:1929] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.109993 8550 log.cc:1199] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/02f6a32db68e4c609da39c3ee6c97348
I20251028 02:59:54.110855 8550 ts_tablet_manager.cc:1950] T 02f6a32db68e4c609da39c3ee6c97348 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.110842 8552 ts_tablet_manager.cc:1929] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.110918 8552 log.cc:1199] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/e4cdbf69eceb4bec9de3368ec959d5f0
I20251028 02:59:54.111155 8552 ts_tablet_manager.cc:1950] T e4cdbf69eceb4bec9de3368ec959d5f0 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.111152 9190 raft_consensus.cc:493] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.111272 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 02f6a32db68e4c609da39c3ee6c97348 (table default.Table_e69b7cea080549b1b8a67cb91703a472 [id=0c94092ef3844117af5e2aa5086dabb6]) successfully deleted
I20251028 02:59:54.111245 9190 raft_consensus.cc:515] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.111727 9190 leader_election.cc:290] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.111805 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet e4cdbf69eceb4bec9de3368ec959d5f0 (table default.Table_50c26dd07fa04d9a838968de2a92f0e5 [id=5290176e2166464799b4d2f4d54e97ff]) successfully deleted
I20251028 02:59:54.112133 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "67c6ceefa3294af8b68150609f46c117" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.112232 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "67c6ceefa3294af8b68150609f46c117" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:54.112488 8356 leader_election.cc:343] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:54.112557 8354 leader_election.cc:343] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.112625 8354 leader_election.cc:304] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.112730 8550 tablet_replica.cc:333] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.112881 8550 raft_consensus.cc:2243] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.112988 8550 raft_consensus.cc:2272] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.113229 8544 tablet_bootstrap.cc:492] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.113245 9190 raft_consensus.cc:2749] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.113317 8544 ts_tablet_manager.cc:1403] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:54.113505 8550 ts_tablet_manager.cc:1916] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.113749 8544 raft_consensus.cc:359] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.113852 8544 raft_consensus.cc:385] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.113878 8544 raft_consensus.cc:740] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.113991 8544 consensus_queue.cc:260] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.114231 8544 ts_tablet_manager.cc:1434] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.114428 8544 tablet_bootstrap.cc:492] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.115767 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet bb0e20b3e05c484394e11bbdea6a44ea from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:54.115773 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 67c6ceefa3294af8b68150609f46c117 consensus state has no leader
I20251028 02:59:54.116537 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_0e8ca626159c41aeb8e360f6e038f7a3 [id=eb0e103661fc4bc7b27bb78fc333fd56]). Expected version 1 got 0
I20251028 02:59:54.116669 7968 catalog_manager.cc:5649] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:54.116912 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 68ca4ff164b24b7498f953ec11cf46be (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:54.117239 8544 tablet_bootstrap.cc:654] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.118113 8552 tablet_replica.cc:333] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.118283 8552 raft_consensus.cc:2243] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.118407 8552 raft_consensus.cc:2272] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.118680 8119 tablet_service.cc:1505] Processing CreateTablet for tablet fcab55fd49da41f7a95d46d76a8dac8b (DEFAULT_TABLE table=default.Table_1d2bfae88392496383bd49d6f3dd832f [id=86e0c4d129fc42f1998b2b0f2dbc2135]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.118933 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcab55fd49da41f7a95d46d76a8dac8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.119455 8396 tablet_service.cc:1505] Processing CreateTablet for tablet fcab55fd49da41f7a95d46d76a8dac8b (DEFAULT_TABLE table=default.Table_1d2bfae88392496383bd49d6f3dd832f [id=86e0c4d129fc42f1998b2b0f2dbc2135]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.119692 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcab55fd49da41f7a95d46d76a8dac8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.119721 8258 tablet_service.cc:1505] Processing CreateTablet for tablet fcab55fd49da41f7a95d46d76a8dac8b (DEFAULT_TABLE table=default.Table_1d2bfae88392496383bd49d6f3dd832f [id=86e0c4d129fc42f1998b2b0f2dbc2135]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.119990 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcab55fd49da41f7a95d46d76a8dac8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.120366 8550 ts_tablet_manager.cc:1929] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.120429 8550 log.cc:1199] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a923402bbc734a0cb44aaf590d1e68ff
I20251028 02:59:54.120626 8550 ts_tablet_manager.cc:1950] T a923402bbc734a0cb44aaf590d1e68ff P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.121923 8116 tablet_service.cc:1505] Processing CreateTablet for tablet cbecb6472dc54989beca91ca7bcb0b57 (DEFAULT_TABLE table=default.Table_3292bf8a27264e8491009af851138333 [id=d317a63dea224c45b4dfd1a205ae4c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.122140 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbecb6472dc54989beca91ca7bcb0b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.122556 8396 tablet_service.cc:1505] Processing CreateTablet for tablet cbecb6472dc54989beca91ca7bcb0b57 (DEFAULT_TABLE table=default.Table_3292bf8a27264e8491009af851138333 [id=d317a63dea224c45b4dfd1a205ae4c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.122571 8259 tablet_service.cc:1505] Processing CreateTablet for tablet cbecb6472dc54989beca91ca7bcb0b57 (DEFAULT_TABLE table=default.Table_3292bf8a27264e8491009af851138333 [id=d317a63dea224c45b4dfd1a205ae4c68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.122776 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbecb6472dc54989beca91ca7bcb0b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.122819 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbecb6472dc54989beca91ca7bcb0b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.125047 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 9a290b8114844aa89b33c75fffc8e1e7 (DEFAULT_TABLE table=default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.125284 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a290b8114844aa89b33c75fffc8e1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.125669 8545 tablet_bootstrap.cc:492] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.125766 8545 ts_tablet_manager.cc:1403] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.095s user 0.002s sys 0.000s
I20251028 02:59:54.126209 8545 raft_consensus.cc:359] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.126310 8545 raft_consensus.cc:385] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.126341 8545 raft_consensus.cc:740] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.126462 8545 consensus_queue.cc:260] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.126739 8545 ts_tablet_manager.cc:1434] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.127198 8545 tablet_bootstrap.cc:492] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.128165 8544 tablet_bootstrap.cc:492] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.128237 8545 tablet_bootstrap.cc:654] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.128259 8544 ts_tablet_manager.cc:1403] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.002s
I20251028 02:59:54.128719 8544 raft_consensus.cc:359] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.128835 8544 raft_consensus.cc:385] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.128870 8544 raft_consensus.cc:740] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.128866 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 7528e7ff127c475789eda805467f0b41 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.128991 8544 consensus_queue.cc:260] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.129262 8544 ts_tablet_manager.cc:1434] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:54.129434 8544 tablet_bootstrap.cc:492] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.130417 8544 tablet_bootstrap.cc:654] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.131397 8546 tablet_bootstrap.cc:492] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.131492 8546 ts_tablet_manager.cc:1403] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.055s user 0.001s sys 0.001s
I20251028 02:59:54.131479 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 5428db004de74f7492c9b8999e53020e (DEFAULT_TABLE table=default.Table_663247205d844151bccc25218dcf39a5 [id=d40c0ee824ba423395e475f68276b5d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.131713 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5428db004de74f7492c9b8999e53020e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.132002 8546 raft_consensus.cc:359] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.132120 8546 raft_consensus.cc:385] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.132129 9190 raft_consensus.cc:493] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.132159 8546 raft_consensus.cc:740] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.132210 9190 raft_consensus.cc:515] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.132297 8546 consensus_queue.cc:260] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.132539 9190 leader_election.cc:290] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.132611 8546 ts_tablet_manager.cc:1434] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.133687 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 9a290b8114844aa89b33c75fffc8e1e7 (DEFAULT_TABLE table=default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.133978 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a290b8114844aa89b33c75fffc8e1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.133982 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4ff8c974231f4b8aa18c18a05ad3e738" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:54.134327 8354 leader_election.cc:343] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.134320 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4ff8c974231f4b8aa18c18a05ad3e738" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.134811 8562 raft_consensus.cc:493] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.134889 8562 raft_consensus.cc:515] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.135180 8562 leader_election.cc:290] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 02:59:54.135875 8356 leader_election.cc:343] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.135946 8356 leader_election.cc:304] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.136204 9190 raft_consensus.cc:2749] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.136569 8552 ts_tablet_manager.cc:1916] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.136796 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 9a290b8114844aa89b33c75fffc8e1e7 (DEFAULT_TABLE table=default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.137136 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a290b8114844aa89b33c75fffc8e1e7. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:54.138574 8083 leader_election.cc:343] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: 03959a61996b40d8b9ef8b4b9708f017
W20251028 02:59:54.138844 8083 leader_election.cc:343] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 03959a61996b40d8b9ef8b4b9708f017
I20251028 02:59:54.138918 8083 leader_election.cc:304] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:54.139096 8552 ts_tablet_manager.cc:1929] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.139145 8562 raft_consensus.cc:2749] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.139284 8552 log.cc:1199] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/1e9234b72f4448089fe7ff6ba49483b7
I20251028 02:59:54.139496 8546 tablet_bootstrap.cc:492] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.139567 8552 ts_tablet_manager.cc:1950] T 1e9234b72f4448089fe7ff6ba49483b7 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.140252 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 1e9234b72f4448089fe7ff6ba49483b7 (table default.Table_3d8776d793a54bd795fb38bd45442f51 [id=be1f381c3ca048d891f5d175fa72fc65]) successfully deleted
I20251028 02:59:54.140496 8546 tablet_bootstrap.cc:654] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.141276 8544 tablet_bootstrap.cc:492] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.141371 8544 ts_tablet_manager.cc:1403] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20251028 02:59:54.141492 8546 tablet_bootstrap.cc:492] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.141583 8546 ts_tablet_manager.cc:1403] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:54.141860 8544 raft_consensus.cc:359] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.141988 8544 raft_consensus.cc:385] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.142019 8544 raft_consensus.cc:740] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.142102 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a923402bbc734a0cb44aaf590d1e68ff (table default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]) successfully deleted
I20251028 02:59:54.142131 8544 consensus_queue.cc:260] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.142416 8544 ts_tablet_manager.cc:1434] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.142612 8544 tablet_bootstrap.cc:492] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.143636 8544 tablet_bootstrap.cc:654] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.143616 7966 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 67c6ceefa3294af8b68150609f46c117 (table default.Table_f9254be0618c49f8a6a36fb9fc10354a [id=86c2f59df3dd4d2195d18835dceff79b]). Expected version 1 got 0
I20251028 02:59:54.143813 7966 catalog_manager.cc:5649] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:54.144044 7966 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_0e8ca626159c41aeb8e360f6e038f7a3 [id=eb0e103661fc4bc7b27bb78fc333fd56]). Expected version 1 got 0
I20251028 02:59:54.142421 8550 tablet_replica.cc:333] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.144541 8550 raft_consensus.cc:2243] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.144644 8550 raft_consensus.cc:2272] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.144965 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_8c2ee567a0b74836997f0d500266c256"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.145321 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c2ee567a0b74836997f0d500266c256 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.146481 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c20f674e0d9a4178a66f16d31bc99638"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.146603 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc] alter complete (version 1)
I20251028 02:59:54.146729 8550 ts_tablet_manager.cc:1916] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:54.146855 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c20f674e0d9a4178a66f16d31bc99638 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.148339 8546 raft_consensus.cc:359] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.148468 8546 raft_consensus.cc:385] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.148505 8546 raft_consensus.cc:740] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.148650 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet cf90b539dc8f47b0947d4e2ca859668a with delete_type TABLET_DATA_DELETED (replaced by tablet 4246c4bd2abf4569987ca346996445d4) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.148861 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet 002de8aede994720b8163f41938af6c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.149194 8552 tablet_replica.cc:333] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.149363 8552 raft_consensus.cc:2243] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.149505 8552 raft_consensus.cc:2272] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.150027 8546 consensus_queue.cc:260] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.150130 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet 4246c4bd2abf4569987ca346996445d4 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.150493 8546 ts_tablet_manager.cc:1434] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:54.150686 8546 tablet_bootstrap.cc:492] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.151360 8550 ts_tablet_manager.cc:1929] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.151487 8550 log.cc:1199] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/aec9427ed4284d66b9bd46c0e1c8774c
I20251028 02:59:54.144289 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 5428db004de74f7492c9b8999e53020e (DEFAULT_TABLE table=default.Table_663247205d844151bccc25218dcf39a5 [id=d40c0ee824ba423395e475f68276b5d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.151770 8550 ts_tablet_manager.cc:1950] T aec9427ed4284d66b9bd46c0e1c8774c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.151985 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5428db004de74f7492c9b8999e53020e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.152319 8546 tablet_bootstrap.cc:654] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.152410 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet aec9427ed4284d66b9bd46c0e1c8774c (table default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]) successfully deleted
I20251028 02:59:54.152627 8550 tablet_replica.cc:333] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.152762 8550 raft_consensus.cc:2243] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.152875 8550 raft_consensus.cc:2272] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.153311 8550 ts_tablet_manager.cc:1916] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.153712 8552 ts_tablet_manager.cc:1916] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.155272 8550 ts_tablet_manager.cc:1929] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.155370 8550 log.cc:1199] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/03959a61996b40d8b9ef8b4b9708f017
I20251028 02:59:54.155599 8550 ts_tablet_manager.cc:1950] T 03959a61996b40d8b9ef8b4b9708f017 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.156200 8552 ts_tablet_manager.cc:1929] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.156271 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 03959a61996b40d8b9ef8b4b9708f017 (table default.Table_5da94c10f01846a59e4ce4ff530330d3 [id=bc83d2963bb74c2d9b914dee53bcf6ba]) successfully deleted
I20251028 02:59:54.156316 8552 log.cc:1199] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/cf90b539dc8f47b0947d4e2ca859668a
I20251028 02:59:54.156451 8546 tablet_bootstrap.cc:492] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.156509 8550 tablet_replica.cc:333] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.156546 8546 ts_tablet_manager.cc:1403] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20251028 02:59:54.156633 8550 raft_consensus.cc:2243] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.156731 8550 raft_consensus.cc:2272] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.156854 8552 ts_tablet_manager.cc:1950] T cf90b539dc8f47b0947d4e2ca859668a P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.157124 8546 raft_consensus.cc:359] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.157188 8550 ts_tablet_manager.cc:1916] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.157243 8546 raft_consensus.cc:385] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.157284 8546 raft_consensus.cc:740] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.157456 8546 consensus_queue.cc:260] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.157873 8546 ts_tablet_manager.cc:1434] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:54.157904 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet cf90b539dc8f47b0947d4e2ca859668a (table default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]) successfully deleted
I20251028 02:59:54.158080 8546 tablet_bootstrap.cc:492] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.158417 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bd4137d37a9847a38ea0d06c10c70186" } new_table_name: "default.Table_a044870c006d4bde90c803732430cbe5" modify_external_catalogs: true
I20251028 02:59:54.159140 8550 ts_tablet_manager.cc:1929] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.159200 8550 log.cc:1199] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/002de8aede994720b8163f41938af6c4
I20251028 02:59:54.159446 8550 ts_tablet_manager.cc:1950] T 002de8aede994720b8163f41938af6c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.160022 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 002de8aede994720b8163f41938af6c4 (table default.Table_c3d6504130c54bdeade5bb23b6cd0679 [id=371c4929d59942f79f41ad071b61e017]) successfully deleted
I20251028 02:59:54.162762 8546 tablet_bootstrap.cc:654] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.162871 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet cdd92cd8a5194e22a91e59aaa4793135
I20251028 02:59:54.163271 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet bb0e20b3e05c484394e11bbdea6a44ea
I20251028 02:59:54.163568 7967 catalog_manager.cc:6719] ReplaceTablet: tablet bb0e20b3e05c484394e11bbdea6a44ea deleted and replaced by tablet bd50e7d4e28541de9799833c53fe7c02
I20251028 02:59:54.163741 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_152433eb6e11487198b81047f13cc1f1" } modify_external_catalogs: true
I20251028 02:59:54.163851 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_152433eb6e11487198b81047f13cc1f1" } modify_external_catalogs: true
I20251028 02:59:54.163978 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet cdd92cd8a5194e22a91e59aaa4793135 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.164129 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0e20b3e05c484394e11bbdea6a44ea with delete_type TABLET_DATA_DELETED (replaced by tablet bd50e7d4e28541de9799833c53fe7c02) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.164523 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0e20b3e05c484394e11bbdea6a44ea with delete_type TABLET_DATA_DELETED (replaced by tablet bd50e7d4e28541de9799833c53fe7c02) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.164657 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1b0dd83e47164c4da9d0b2311dfffc81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.165038 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b0dd83e47164c4da9d0b2311dfffc81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.165037 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet cdd92cd8a5194e22a91e59aaa4793135 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.166978 8116 tablet_service.cc:1505] Processing CreateTablet for tablet a808098c70b54ad3a0230352af0b743c (DEFAULT_TABLE table=default.Table_f2b0692abe27469a8f0728978e62ae26 [id=5dd4cb94fa384facbdd1aef677a4b498]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.167366 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a808098c70b54ad3a0230352af0b743c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.167575 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet cdd92cd8a5194e22a91e59aaa4793135 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.169799 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 9583046f549d411a9e341560f428381c (DEFAULT_TABLE table=default.Table_8b7624342986474fbd0e31c86947f781 [id=530470c553154d76bf0df08ddfbf2fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.170045 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9583046f549d411a9e341560f428381c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.170276 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4246c4bd2abf4569987ca346996445d4
I20251028 02:59:54.170552 7962 catalog_manager.cc:6719] ReplaceTablet: tablet 4246c4bd2abf4569987ca346996445d4 deleted and replaced by tablet 81b521c9f57844bb9873f6b26b5cf661
I20251028 02:59:54.170277 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_0e8ca626159c41aeb8e360f6e038f7a3 [id=eb0e103661fc4bc7b27bb78fc333fd56]). Expected version 1 got 0
I20251028 02:59:54.171401 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (table default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]). Expected version 1 got 0
I20251028 02:59:54.171891 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 4246c4bd2abf4569987ca346996445d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 81b521c9f57844bb9873f6b26b5cf661) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.172410 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a8331b21af69463cb7589619e784051d
I20251028 02:59:54.172484 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 529b865389f94471a6849d17ff206e4c (DEFAULT_TABLE table=default.Table_153d49d056a3458a9565c70d954187ae [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.172693 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 529b865389f94471a6849d17ff206e4c. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:54.171427 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c0f428e5c6fe43fc8f78761d868ddd09 has no consensus state
I20251028 02:59:54.174042 8545 tablet_bootstrap.cc:492] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.174137 8545 ts_tablet_manager.cc:1403] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20251028 02:59:54.174685 8545 raft_consensus.cc:359] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.174808 8545 raft_consensus.cc:385] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.174844 8545 raft_consensus.cc:740] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.174969 8545 consensus_queue.cc:260] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.175349 8545 ts_tablet_manager.cc:1434] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.175562 8545 tablet_bootstrap.cc:492] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.175671 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0e20b3e05c484394e11bbdea6a44ea with delete_type TABLET_DATA_DELETED (replaced by tablet bd50e7d4e28541de9799833c53fe7c02) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.175748 8119 tablet_service.cc:1552] Processing DeleteTablet for tablet a8331b21af69463cb7589619e784051d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.177302 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 5428db004de74f7492c9b8999e53020e (DEFAULT_TABLE table=default.Table_663247205d844151bccc25218dcf39a5 [id=d40c0ee824ba423395e475f68276b5d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.177544 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5428db004de74f7492c9b8999e53020e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.177860 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 26a344e2e03e4eeaa688275e9ecbf63f
W20251028 02:59:54.179068 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet cdd92cd8a5194e22a91e59aaa4793135 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cdd92cd8a5194e22a91e59aaa4793135 already in progress: creating tablet
W20251028 02:59:54.179206 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet bb0e20b3e05c484394e11bbdea6a44ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb0e20b3e05c484394e11bbdea6a44ea already in progress: creating tablet
W20251028 02:59:54.179342 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4246c4bd2abf4569987ca346996445d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4246c4bd2abf4569987ca346996445d4 already in progress: creating tablet
W20251028 02:59:54.179452 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a8331b21af69463cb7589619e784051d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a8331b21af69463cb7589619e784051d already in progress: creating tablet
I20251028 02:59:54.179450 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 9583046f549d411a9e341560f428381c (DEFAULT_TABLE table=default.Table_8b7624342986474fbd0e31c86947f781 [id=530470c553154d76bf0df08ddfbf2fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.179669 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9583046f549d411a9e341560f428381c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.179834 8400 tablet_service.cc:1505] Processing CreateTablet for tablet a808098c70b54ad3a0230352af0b743c (DEFAULT_TABLE table=default.Table_f2b0692abe27469a8f0728978e62ae26 [id=5dd4cb94fa384facbdd1aef677a4b498]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.180037 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a808098c70b54ad3a0230352af0b743c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.180320 8545 tablet_bootstrap.cc:654] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.181103 8259 tablet_service.cc:1505] Processing CreateTablet for tablet a808098c70b54ad3a0230352af0b743c (DEFAULT_TABLE table=default.Table_f2b0692abe27469a8f0728978e62ae26 [id=5dd4cb94fa384facbdd1aef677a4b498]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.181376 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a808098c70b54ad3a0230352af0b743c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.181712 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet a8331b21af69463cb7589619e784051d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.181839 9190 raft_consensus.cc:493] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.181907 9190 raft_consensus.cc:515] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.182021 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 4246c4bd2abf4569987ca346996445d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 81b521c9f57844bb9873f6b26b5cf661) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.182026 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet a8331b21af69463cb7589619e784051d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.182214 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 26a344e2e03e4eeaa688275e9ecbf63f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.182276 9190 leader_election.cc:290] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.182502 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 9583046f549d411a9e341560f428381c (DEFAULT_TABLE table=default.Table_8b7624342986474fbd0e31c86947f781 [id=530470c553154d76bf0df08ddfbf2fd0]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:54.182606 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 26a344e2e03e4eeaa688275e9ecbf63f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26a344e2e03e4eeaa688275e9ecbf63f already in progress: creating tablet
I20251028 02:59:54.182715 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9583046f549d411a9e341560f428381c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.183003 8552 tablet_replica.cc:333] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.183123 8552 raft_consensus.cc:2243] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.183219 8552 raft_consensus.cc:2272] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.183630 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 4246c4bd2abf4569987ca346996445d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 81b521c9f57844bb9873f6b26b5cf661) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.183696 8545 tablet_bootstrap.cc:492] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.183840 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1e9234b72f4448089fe7ff6ba49483b7" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:54.184301 8356 leader_election.cc:343] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: 1e9234b72f4448089fe7ff6ba49483b7
I20251028 02:59:54.184417 8545 ts_tablet_manager.cc:1403] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 02:59:54.184923 8545 raft_consensus.cc:359] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.185032 8545 raft_consensus.cc:385] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.185083 8545 raft_consensus.cc:740] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.185245 8545 consensus_queue.cc:260] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:54.185475 8354 leader_election.cc:343] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.185539 8354 leader_election.cc:304] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.185684 9190 raft_consensus.cc:2749] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.186585 7966 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_1306f4d4723b45fbb688bfccebd665a5 [id=4eb2743632814baab2e39bbdce700d6b] alter complete (version 4)
I20251028 02:59:54.188472 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 8ae2a510d64243a9b0fb7784ee56d29f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.188858 8545 ts_tablet_manager.cc:1434] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:54.189137 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet edecfb891eaa4a718ef1fdf399e7ef9d with delete_type TABLET_DATA_DELETED (replaced by tablet 7801d57550dc40c39f2d3cf7e11e3713) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.189500 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 7e347497892b4add9f7b4e19bde67878 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.190209 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]). Expected version 1 got 0
I20251028 02:59:54.190305 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 67c6ceefa3294af8b68150609f46c117 (table default.Table_f9254be0618c49f8a6a36fb9fc10354a [id=86c2f59df3dd4d2195d18835dceff79b]). Expected version 1 got 0
I20251028 02:59:54.191200 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 6c87d03fad194ca89f1808da6dbdf0a4 with delete_type TABLET_DATA_DELETED (replaced by tablet a3096fde2ac94ab197ce5e659c4d2be0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.191951 8550 tablet_replica.cc:333] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.192135 8550 raft_consensus.cc:2243] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.192286 8550 raft_consensus.cc:2272] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:54.193023 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet cdd92cd8a5194e22a91e59aaa4793135 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cdd92cd8a5194e22a91e59aaa4793135 already in progress: creating tablet
I20251028 02:59:54.193221 8550 ts_tablet_manager.cc:1916] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.193712 8546 tablet_bootstrap.cc:492] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.193825 8546 ts_tablet_manager.cc:1403] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20251028 02:59:54.193186 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_94d71cfa43134dff9a62343374aed8fe" } new_table_name: "default.Table_d6e8f311ccc94210b7a22755a7b4c21a" modify_external_catalogs: true
I20251028 02:59:54.195672 8545 tablet_bootstrap.cc:492] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:54.193183 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet bb0e20b3e05c484394e11bbdea6a44ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb0e20b3e05c484394e11bbdea6a44ea already in progress: creating tablet
W20251028 02:59:54.196241 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a8331b21af69463cb7589619e784051d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a8331b21af69463cb7589619e784051d already in progress: creating tablet
W20251028 02:59:54.196390 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4246c4bd2abf4569987ca346996445d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4246c4bd2abf4569987ca346996445d4 already in progress: creating tablet
I20251028 02:59:54.196401 8550 ts_tablet_manager.cc:1929] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251028 02:59:54.196497 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet bb0e20b3e05c484394e11bbdea6a44ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb0e20b3e05c484394e11bbdea6a44ea already in progress: creating tablet
I20251028 02:59:54.196609 8550 log.cc:1199] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8ae2a510d64243a9b0fb7784ee56d29f
W20251028 02:59:54.196621 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet cdd92cd8a5194e22a91e59aaa4793135 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cdd92cd8a5194e22a91e59aaa4793135 already in progress: creating tablet
W20251028 02:59:54.196725 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4246c4bd2abf4569987ca346996445d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4246c4bd2abf4569987ca346996445d4 already in progress: creating tablet
I20251028 02:59:54.196949 8550 ts_tablet_manager.cc:1950] T 8ae2a510d64243a9b0fb7784ee56d29f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.197114 8545 tablet_bootstrap.cc:654] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.197204 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 26a344e2e03e4eeaa688275e9ecbf63f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.197695 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8ae2a510d64243a9b0fb7784ee56d29f (table default.Table_23ea93791ab84db8acf0b4e6c21141d8 [id=2fccc00ed79649eabd08b00696a27710]) successfully deleted
I20251028 02:59:54.197937 8550 tablet_replica.cc:333] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.198084 8550 raft_consensus.cc:2243] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.198105 8552 ts_tablet_manager.cc:1916] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.198186 8550 raft_consensus.cc:2272] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.198808 8550 ts_tablet_manager.cc:1916] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.198972 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 7e347497892b4add9f7b4e19bde67878
I20251028 02:59:54.199270 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 7e347497892b4add9f7b4e19bde67878 deleted and replaced by tablet d59acfb0e12643ebbb406dbda7bef29b
I20251028 02:59:54.199478 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e347497892b4add9f7b4e19bde67878 with delete_type TABLET_DATA_DELETED (replaced by tablet d59acfb0e12643ebbb406dbda7bef29b) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.200436 8552 ts_tablet_manager.cc:1929] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.200537 8552 log.cc:1199] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/a8331b21af69463cb7589619e784051d
I20251028 02:59:54.200758 8552 ts_tablet_manager.cc:1950] T a8331b21af69463cb7589619e784051d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.200829 8546 raft_consensus.cc:359] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.200955 8546 raft_consensus.cc:385] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.200992 8546 raft_consensus.cc:740] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.202008 8550 ts_tablet_manager.cc:1929] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.202148 8550 log.cc:1199] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/edecfb891eaa4a718ef1fdf399e7ef9d
I20251028 02:59:54.202481 8550 ts_tablet_manager.cc:1950] T edecfb891eaa4a718ef1fdf399e7ef9d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.202623 9190 raft_consensus.cc:493] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.202739 9190 raft_consensus.cc:515] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.203177 9190 leader_election.cc:290] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.203286 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet edecfb891eaa4a718ef1fdf399e7ef9d (table default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]) successfully deleted
I20251028 02:59:54.203624 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fd9cc1847ff4df7a09e2e698786d515" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.203923 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fd9cc1847ff4df7a09e2e698786d515" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:54.204070 8356 leader_election.cc:343] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.204274 8550 tablet_replica.cc:333] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
W20251028 02:59:54.204284 8354 leader_election.cc:343] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.204365 8354 leader_election.cc:304] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.204428 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 26a344e2e03e4eeaa688275e9ecbf63f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.204566 8550 raft_consensus.cc:2243] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.204707 8550 raft_consensus.cc:2272] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.204823 9190 raft_consensus.cc:2749] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.204864 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e9234b72f4448089fe7ff6ba49483b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.204991 8551 tablet_replica.cc:333] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.205119 8551 raft_consensus.cc:2243] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.205231 8551 raft_consensus.cc:2272] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.204955 8546 consensus_queue.cc:260] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.205308 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet cf90b539dc8f47b0947d4e2ca859668a with delete_type TABLET_DATA_DELETED (replaced by tablet 4246c4bd2abf4569987ca346996445d4) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.205585 8550 ts_tablet_manager.cc:1916] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.205631 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fd9cc1847ff4df7a09e2e698786d515 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.205710 8551 ts_tablet_manager.cc:1916] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.205933 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e347497892b4add9f7b4e19bde67878 with delete_type TABLET_DATA_DELETED (replaced by tablet d59acfb0e12643ebbb406dbda7bef29b) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.206264 8546 ts_tablet_manager.cc:1434] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20251028 02:59:54.206621 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e347497892b4add9f7b4e19bde67878 with delete_type TABLET_DATA_DELETED (replaced by tablet d59acfb0e12643ebbb406dbda7bef29b) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.208837 8546 tablet_bootstrap.cc:492] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.209007 8550 ts_tablet_manager.cc:1929] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.209125 8550 log.cc:1199] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/6c87d03fad194ca89f1808da6dbdf0a4
I20251028 02:59:54.209134 8551 ts_tablet_manager.cc:1929] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.209203 8551 log.cc:1199] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/1e9234b72f4448089fe7ff6ba49483b7
I20251028 02:59:54.209431 8551 ts_tablet_manager.cc:1950] T 1e9234b72f4448089fe7ff6ba49483b7 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.209933 8550 ts_tablet_manager.cc:1950] T 6c87d03fad194ca89f1808da6dbdf0a4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.210485 9011 raft_consensus.cc:493] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.210575 9011 raft_consensus.cc:515] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:54.210611 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 26a344e2e03e4eeaa688275e9ecbf63f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26a344e2e03e4eeaa688275e9ecbf63f already in progress: creating tablet
I20251028 02:59:54.210737 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet a8331b21af69463cb7589619e784051d (table default.Table_f73fbea5f42f4caa815469fbb7504f49 [id=79b638b293df431cbd580e62d24e8238]) successfully deleted
W20251028 02:59:54.210834 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 26a344e2e03e4eeaa688275e9ecbf63f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26a344e2e03e4eeaa688275e9ecbf63f already in progress: creating tablet
I20251028 02:59:54.210933 9011 leader_election.cc:290] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.210948 8551 tablet_replica.cc:333] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.211076 8551 raft_consensus.cc:2243] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.211174 8551 raft_consensus.cc:2272] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.211184 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet edecfb891eaa4a718ef1fdf399e7ef9d with delete_type TABLET_DATA_DELETED (replaced by tablet 7801d57550dc40c39f2d3cf7e11e3713) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.211987 8546 tablet_bootstrap.cc:654] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:54.212332 8220 leader_election.cc:343] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: edecfb891eaa4a718ef1fdf399e7ef9d
I20251028 02:59:54.212831 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 6c87d03fad194ca89f1808da6dbdf0a4 (table default.Table_6d8f283a536e46fcbf3d8abcf9d703bb [id=a6d84958fd654087940c9806e2e3fbc0]) successfully deleted
I20251028 02:59:54.213085 8550 tablet_replica.cc:333] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
W20251028 02:59:54.213111 8218 leader_election.cc:343] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: edecfb891eaa4a718ef1fdf399e7ef9d
I20251028 02:59:54.213191 8218 leader_election.cc:304] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.213344 8550 raft_consensus.cc:2243] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.213399 9011 raft_consensus.cc:2749] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.213526 8550 raft_consensus.cc:2272] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.213953 8551 ts_tablet_manager.cc:1916] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.214124 8550 ts_tablet_manager.cc:1916] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.215646 8551 ts_tablet_manager.cc:1929] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.215709 8551 log.cc:1199] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/cf90b539dc8f47b0947d4e2ca859668a
I20251028 02:59:54.215871 8551 ts_tablet_manager.cc:1950] T cf90b539dc8f47b0947d4e2ca859668a P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.216341 8551 tablet_replica.cc:333] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.216473 8551 raft_consensus.cc:2243] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.216480 8550 ts_tablet_manager.cc:1929] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:54.216557 8551 raft_consensus.cc:2272] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.217020 8551 ts_tablet_manager.cc:1916] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.217168 8552 tablet_replica.cc:333] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.217324 8552 raft_consensus.cc:2243] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.217483 8552 raft_consensus.cc:2272] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.217903 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 1e9234b72f4448089fe7ff6ba49483b7 (table default.Table_3d8776d793a54bd795fb38bd45442f51 [id=be1f381c3ca048d891f5d175fa72fc65]) successfully deleted
I20251028 02:59:54.218343 9048 tablet.cc:1722] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:54.218549 8550 log.cc:1199] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/7e347497892b4add9f7b4e19bde67878
I20251028 02:59:54.218811 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet cf90b539dc8f47b0947d4e2ca859668a (table default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]) successfully deleted
I20251028 02:59:54.218876 9023 tablet.cc:1722] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:54.219130 9122 tablet.cc:1722] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:54.219913 8552 ts_tablet_manager.cc:1916] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.219980 8550 ts_tablet_manager.cc:1950] T 7e347497892b4add9f7b4e19bde67878 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.220669 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 7e347497892b4add9f7b4e19bde67878 (table default.Table_e4f1369ad6ad4110b5854544d4a266bd [id=ccea14281b2c420a99cede953a8de44d]) successfully deleted
I20251028 02:59:54.220890 8551 ts_tablet_manager.cc:1929] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.220961 8551 log.cc:1199] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/5fd9cc1847ff4df7a09e2e698786d515
I20251028 02:59:54.221921 8552 ts_tablet_manager.cc:1929] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:54.221988 8552 log.cc:1199] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/7e347497892b4add9f7b4e19bde67878
I20251028 02:59:54.222222 8552 ts_tablet_manager.cc:1950] T 7e347497892b4add9f7b4e19bde67878 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.225014 8551 ts_tablet_manager.cc:1950] T 5fd9cc1847ff4df7a09e2e698786d515 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.227217 8551 tablet_replica.cc:333] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.227396 8551 raft_consensus.cc:2243] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:54.227564 8551 raft_consensus.cc:2272] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.228165 8551 ts_tablet_manager.cc:1916] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.229585 7967 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_d6e8f311ccc94210b7a22755a7b4c21a [id=7f26fae434ec4bdda077c8ceb5350aac] alter complete (version 2)
I20251028 02:59:54.231648 8551 ts_tablet_manager.cc:1929] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:54.231730 8551 log.cc:1199] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/7e347497892b4add9f7b4e19bde67878
I20251028 02:59:54.232008 8551 ts_tablet_manager.cc:1950] T 7e347497892b4add9f7b4e19bde67878 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.232048 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 529b865389f94471a6849d17ff206e4c (DEFAULT_TABLE table=default.Table_153d49d056a3458a9565c70d954187ae [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.232048 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 529b865389f94471a6849d17ff206e4c (DEFAULT_TABLE table=default.Table_153d49d056a3458a9565c70d954187ae [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.232331 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 529b865389f94471a6849d17ff206e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.232367 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 529b865389f94471a6849d17ff206e4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.234265 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 5fd9cc1847ff4df7a09e2e698786d515 (table default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]) successfully deleted
I20251028 02:59:54.234391 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 7e347497892b4add9f7b4e19bde67878 (table default.Table_e4f1369ad6ad4110b5854544d4a266bd [id=ccea14281b2c420a99cede953a8de44d]) successfully deleted
I20251028 02:59:54.235318 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 7e347497892b4add9f7b4e19bde67878 (table default.Table_e4f1369ad6ad4110b5854544d4a266bd [id=ccea14281b2c420a99cede953a8de44d]) successfully deleted
I20251028 02:59:54.235492 8420 raft_consensus.cc:1275] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:54.235625 8147 raft_consensus.cc:1275] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:54.236014 8965 consensus_queue.cc:1048] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.236286 8965 consensus_queue.cc:1048] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.237248 8544 tablet_bootstrap.cc:492] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.237370 8544 ts_tablet_manager.cc:1403] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.095s user 0.002s sys 0.000s
I20251028 02:59:54.237852 8544 raft_consensus.cc:359] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.237951 8544 raft_consensus.cc:385] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.237982 8544 raft_consensus.cc:740] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.238113 8544 consensus_queue.cc:260] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.238400 8544 ts_tablet_manager.cc:1434] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.238602 8544 tablet_bootstrap.cc:492] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.239941 8544 tablet_bootstrap.cc:654] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.240240 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]). Expected version 1 got 0
I20251028 02:59:54.241757 8545 tablet_bootstrap.cc:492] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.242185 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]). Expected version 1 got 0
I20251028 02:59:54.242878 8545 ts_tablet_manager.cc:1403] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20251028 02:59:54.243187 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet e4cdbf69eceb4bec9de3368ec959d5f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 74120decd53d4de19c5f6fedf70ecaae) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.243559 8550 tablet_replica.cc:333] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.243511 8545 raft_consensus.cc:359] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.243618 8545 raft_consensus.cc:385] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.243661 8545 raft_consensus.cc:740] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.243691 8550 raft_consensus.cc:2243] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.243813 8550 raft_consensus.cc:2272] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.243829 8545 consensus_queue.cc:260] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.245502 9122 tablet.cc:1722] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.246008 9023 tablet.cc:1722] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.248088 9048 tablet.cc:1722] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.249313 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]). Expected version 1 got 0
I20251028 02:59:54.251334 7968 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]). Expected version 1 got 0
I20251028 02:59:54.251830 8545 ts_tablet_manager.cc:1434] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20251028 02:59:54.252102 8545 tablet_bootstrap.cc:492] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.254196 8545 tablet_bootstrap.cc:654] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.256232 8550 ts_tablet_manager.cc:1916] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.257292 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet a8331b21af69463cb7589619e784051d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.257833 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_635064f8d8484a978cd3e389004f2969"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.258284 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_635064f8d8484a978cd3e389004f2969 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.258677 8550 ts_tablet_manager.cc:1929] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.258746 8550 log.cc:1199] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/e4cdbf69eceb4bec9de3368ec959d5f0
I20251028 02:59:54.258965 8550 ts_tablet_manager.cc:1950] T e4cdbf69eceb4bec9de3368ec959d5f0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.259639 8551 tablet_replica.cc:333] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.259814 8551 raft_consensus.cc:2243] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.259934 8551 raft_consensus.cc:2272] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.260509 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet e4cdbf69eceb4bec9de3368ec959d5f0 (table default.Table_50c26dd07fa04d9a838968de2a92f0e5 [id=5290176e2166464799b4d2f4d54e97ff]) successfully deleted
I20251028 02:59:54.261519 8552 tablet_replica.cc:333] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.261710 8552 raft_consensus.cc:2243] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.261890 8552 raft_consensus.cc:2272] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.262742 8551 ts_tablet_manager.cc:1916] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.263536 8552 ts_tablet_manager.cc:1916] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.264938 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ad3466e031ae40059b8e3ec01c7ed5a9" } modify_external_catalogs: true
I20251028 02:59:54.265132 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ad3466e031ae40059b8e3ec01c7ed5a9" } modify_external_catalogs: true
I20251028 02:59:54.265964 8551 ts_tablet_manager.cc:1929] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.266459 8544 tablet_bootstrap.cc:492] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.266554 8544 ts_tablet_manager.cc:1403] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.002s
I20251028 02:59:54.266865 8551 log.cc:1199] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a8331b21af69463cb7589619e784051d
I20251028 02:59:54.267052 8544 raft_consensus.cc:359] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.267174 8544 raft_consensus.cc:385] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.267216 8544 raft_consensus.cc:740] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.267405 8551 ts_tablet_manager.cc:1950] T a8331b21af69463cb7589619e784051d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.267387 8544 consensus_queue.cc:260] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.267827 8965 raft_consensus.cc:493] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.267920 8965 raft_consensus.cc:515] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.268322 8965 leader_election.cc:290] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.268558 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a8331b21af69463cb7589619e784051d (table default.Table_f73fbea5f42f4caa815469fbb7504f49 [id=79b638b293df431cbd580e62d24e8238]) successfully deleted
I20251028 02:59:54.269137 8544 ts_tablet_manager.cc:1434] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251028 02:59:54.269230 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7165de68f6c4987a41cd33b5c6298c4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:54.269891 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 4bcdf0368e394d649086042f1fe9774b from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.270133 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a3c391d8f9a14324b6ae88bd860a0aa9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.270339 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_65ab6aba20224249bcf1b91e6c9c65dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.270542 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a3c391d8f9a14324b6ae88bd860a0aa9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:54.270570 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65ab6aba20224249bcf1b91e6c9c65dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.270849 8545 tablet_bootstrap.cc:492] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.269358 8420 raft_consensus.cc:2468] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:54.270937 8545 ts_tablet_manager.cc:1403] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20251028 02:59:54.271198 8220 leader_election.cc:304] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:54.271399 8965 raft_consensus.cc:2804] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:54.271389 7967 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672] alter complete (version 1)
I20251028 02:59:54.271454 8965 raft_consensus.cc:493] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:54.271498 8965 raft_consensus.cc:3060] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.271497 8545 raft_consensus.cc:359] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.271852 8545 raft_consensus.cc:385] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.271895 8545 raft_consensus.cc:740] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.272157 8545 consensus_queue.cc:260] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.272565 8965 raft_consensus.cc:515] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.272933 8965 leader_election.cc:290] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.273115 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7165de68f6c4987a41cd33b5c6298c4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:54.273227 8544 tablet_bootstrap.cc:492] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.273234 8147 raft_consensus.cc:2468] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:54.274353 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7165de68f6c4987a41cd33b5c6298c4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:54.274461 8147 raft_consensus.cc:3060] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.274546 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7165de68f6c4987a41cd33b5c6298c4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:54.274642 8420 raft_consensus.cc:3060] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.275497 8147 raft_consensus.cc:2468] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:54.275763 8420 raft_consensus.cc:2468] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:54.276091 8220 leader_election.cc:304] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:54.276307 8965 raft_consensus.cc:2804] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:54.276400 8965 raft_consensus.cc:697] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:54.276543 8965 consensus_queue.cc:237] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.277191 8545 ts_tablet_manager.cc:1434] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251028 02:59:54.277442 8545 tablet_bootstrap.cc:492] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.278662 8544 tablet_bootstrap.cc:654] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.278852 8545 tablet_bootstrap.cc:654] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.284353 8544 tablet_bootstrap.cc:492] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.284468 8544 ts_tablet_manager.cc:1403] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20251028 02:59:54.284512 8552 ts_tablet_manager.cc:1929] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.284591 8552 log.cc:1199] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/edecfb891eaa4a718ef1fdf399e7ef9d
I20251028 02:59:54.284833 8552 ts_tablet_manager.cc:1950] T edecfb891eaa4a718ef1fdf399e7ef9d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.284986 8544 raft_consensus.cc:359] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.285108 8544 raft_consensus.cc:385] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.285146 8544 raft_consensus.cc:740] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.285316 8544 consensus_queue.cc:260] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.285920 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_00a06d6bfdff457ab18a0404c944f780"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.286393 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6d8f283a536e46fcbf3d8abcf9d703bb" } modify_external_catalogs: true
W20251028 02:59:54.286455 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00a06d6bfdff457ab18a0404c944f780 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.286505 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6d8f283a536e46fcbf3d8abcf9d703bb" } modify_external_catalogs: true
I20251028 02:59:54.287638 8544 ts_tablet_manager.cc:1434] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251028 02:59:54.289215 8544 tablet_bootstrap.cc:492] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.290534 8544 tablet_bootstrap.cc:654] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.290472 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bc089d8155d4463082112a190f9da5b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.290912 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc089d8155d4463082112a190f9da5b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.291754 8544 tablet_bootstrap.cc:492] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.291839 8544 ts_tablet_manager.cc:1403] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20251028 02:59:54.292420 8544 raft_consensus.cc:359] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.292510 8544 raft_consensus.cc:385] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.292536 8544 raft_consensus.cc:740] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.292688 8544 consensus_queue.cc:260] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.295127 8544 ts_tablet_manager.cc:1434] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:54.295855 8544 tablet_bootstrap.cc:492] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.297528 8544 tablet_bootstrap.cc:654] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.298471 8545 tablet_bootstrap.cc:492] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.298597 8545 ts_tablet_manager.cc:1403] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20251028 02:59:54.299119 8545 raft_consensus.cc:359] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.299237 8545 raft_consensus.cc:385] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.299268 8545 raft_consensus.cc:740] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.299448 8545 consensus_queue.cc:260] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.299759 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4457c9f159ad428c99a082b56e95f8ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.300122 8546 tablet_bootstrap.cc:492] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.300217 8546 ts_tablet_manager.cc:1403] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.092s user 0.001s sys 0.000s
W20251028 02:59:54.300246 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4457c9f159ad428c99a082b56e95f8ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.300932 8545 ts_tablet_manager.cc:1434] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251028 02:59:54.300884 8546 raft_consensus.cc:359] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.300987 8546 raft_consensus.cc:385] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.301019 8546 raft_consensus.cc:740] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.301132 8545 tablet_bootstrap.cc:492] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.301282 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_05d02074a9864884801e6bbd08265546" } new_table_name: "default.Table_d802b3738006457a9d34cd51d2663e92" modify_external_catalogs: true
I20251028 02:59:54.302549 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet edecfb891eaa4a718ef1fdf399e7ef9d (table default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]) successfully deleted
I20251028 02:59:54.303231 8545 tablet_bootstrap.cc:654] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.306315 8546 consensus_queue.cc:260] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.306675 8546 ts_tablet_manager.cc:1434] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251028 02:59:54.306869 8546 tablet_bootstrap.cc:492] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.310945 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b60ad574ddda4f7d9c66bf24dd5c612a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.311757 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c510beb7614e484c8b45ff008bca293c" } new_table_name: "default.Table_176b16f7b1fd4a30991b82efd4ddd497" modify_external_catalogs: true
I20251028 02:59:54.312506 7963 catalog_manager.cc:5649] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:54.313827 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a (DEFAULT_TABLE table=default.Table_1b0dd83e47164c4da9d0b2311dfffc81 [id=6d26ce36ea3a479c9397a1e6ce995d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.314167 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.314328 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a (DEFAULT_TABLE table=default.Table_1b0dd83e47164c4da9d0b2311dfffc81 [id=6d26ce36ea3a479c9397a1e6ce995d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.314594 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.316847 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 6dfc5240893546d7a6a5397cd08044dd (DEFAULT_TABLE table=default.Table_c20f674e0d9a4178a66f16d31bc99638 [id=49d0cfc3409f4e2d84b5e1b2634193e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.317070 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dfc5240893546d7a6a5397cd08044dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.317665 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 6dfc5240893546d7a6a5397cd08044dd (DEFAULT_TABLE table=default.Table_c20f674e0d9a4178a66f16d31bc99638 [id=49d0cfc3409f4e2d84b5e1b2634193e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.317899 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dfc5240893546d7a6a5397cd08044dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.319850 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a (DEFAULT_TABLE table=default.Table_1b0dd83e47164c4da9d0b2311dfffc81 [id=6d26ce36ea3a479c9397a1e6ce995d68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.320250 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 6dfc5240893546d7a6a5397cd08044dd (DEFAULT_TABLE table=default.Table_c20f674e0d9a4178a66f16d31bc99638 [id=49d0cfc3409f4e2d84b5e1b2634193e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.320492 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 09d65eef81b2421c92678015b6da4fbb (DEFAULT_TABLE table=default.Table_d611f94943f44256ad3b9ca490f3282c [id=6d4246a1b1ea4448b8fbf6a5abecfed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.320676 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09d65eef81b2421c92678015b6da4fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.321846 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 09d65eef81b2421c92678015b6da4fbb (DEFAULT_TABLE table=default.Table_d611f94943f44256ad3b9ca490f3282c [id=6d4246a1b1ea4448b8fbf6a5abecfed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.321938 8117 tablet_service.cc:1505] Processing CreateTablet for tablet f6ac7b5fff0848cebf710ba3ed50def4 (DEFAULT_TABLE table=default.Table_bd14742bd9f943cc93df997dfecd2719 [id=6f3e689a22ae4160ae7537d955c06a90]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.322083 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09d65eef81b2421c92678015b6da4fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.322186 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6ac7b5fff0848cebf710ba3ed50def4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.322333 8965 raft_consensus.cc:493] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.322373 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 874b6660d86b48ebad03ecc54588da28 (DEFAULT_TABLE table=default.Table_c7e0f5beff234f17b69504efc678924d [id=181b50c2d29a4b17b8189e3abb10a420]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.322427 8965 raft_consensus.cc:515] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.322578 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 874b6660d86b48ebad03ecc54588da28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.322844 8965 leader_election.cc:290] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:54.323768 8220 leader_election.cc:343] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:54.324352 8123 tablet_service.cc:1505] Processing CreateTablet for tablet c0f428e5c6fe43fc8f78761d868ddd09 (DEFAULT_TABLE table=default.Table_a044870c006d4bde90c803732430cbe5 [id=0ee1306a57fd419497c97f844e58157e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.324492 8122 tablet_service.cc:1505] Processing CreateTablet for tablet bd50e7d4e28541de9799833c53fe7c02 (DEFAULT_TABLE table=default.Table_424dccd707fc4fa1bd9d01e78cab2f92 [id=dbe3cee1ce8a47fab35d8303c2950b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.324548 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f428e5c6fe43fc8f78761d868ddd09. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.324683 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd50e7d4e28541de9799833c53fe7c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.324728 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 322cbd8253ce45118ad14e8f8f2f1f74 (DEFAULT_TABLE table=default.Table_610f127f3f9348309aa1ce4b6537e5b1 [id=40f8798a68d54ea991ef9bf6887dfe36]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.324908 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 322cbd8253ce45118ad14e8f8f2f1f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.324873 8120 tablet_service.cc:1505] Processing CreateTablet for tablet b46554b0df894beb8141126aa7ee7b8e (DEFAULT_TABLE table=default.Table_c510beb7614e484c8b45ff008bca293c [id=9e5ad11a9ea34a3a8c629f48142b68f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.325063 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46554b0df894beb8141126aa7ee7b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.326642 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 322cbd8253ce45118ad14e8f8f2f1f74 (DEFAULT_TABLE table=default.Table_610f127f3f9348309aa1ce4b6537e5b1 [id=40f8798a68d54ea991ef9bf6887dfe36]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.326733 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 7b552f63895449aea79cbd8867cc063c (DEFAULT_TABLE table=default.Table_8c2ee567a0b74836997f0d500266c256 [id=efaa3797242b4a238c7aac2d7427451c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.326944 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b552f63895449aea79cbd8867cc063c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.327013 8262 tablet_service.cc:1505] Processing CreateTablet for tablet f6ac7b5fff0848cebf710ba3ed50def4 (DEFAULT_TABLE table=default.Table_bd14742bd9f943cc93df997dfecd2719 [id=6f3e689a22ae4160ae7537d955c06a90]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.327214 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 874b6660d86b48ebad03ecc54588da28 (DEFAULT_TABLE table=default.Table_c7e0f5beff234f17b69504efc678924d [id=181b50c2d29a4b17b8189e3abb10a420]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:54.327324 8218 leader_election.cc:343] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:54.327389 8218 leader_election.cc:304] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.327454 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 874b6660d86b48ebad03ecc54588da28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.327487 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 81b521c9f57844bb9873f6b26b5cf661 (DEFAULT_TABLE table=default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.327682 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b521c9f57844bb9873f6b26b5cf661. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.328924 8965 raft_consensus.cc:2749] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.329492 8394 tablet_service.cc:1505] Processing CreateTablet for tablet bd50e7d4e28541de9799833c53fe7c02 (DEFAULT_TABLE table=default.Table_424dccd707fc4fa1bd9d01e78cab2f92 [id=dbe3cee1ce8a47fab35d8303c2950b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.329665 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd50e7d4e28541de9799833c53fe7c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.329663 8127 tablet_service.cc:1505] Processing CreateTablet for tablet f08a6e30211a4d59934c36c7a26ac983 (DEFAULT_TABLE table=default.Table_2de1f8c1f3564959a40963351d7d60b6 [id=b9c53ab5608f49309950ed79b8272710]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.329905 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f08a6e30211a4d59934c36c7a26ac983. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.331441 8265 tablet_service.cc:1505] Processing CreateTablet for tablet bd50e7d4e28541de9799833c53fe7c02 (DEFAULT_TABLE table=default.Table_424dccd707fc4fa1bd9d01e78cab2f92 [id=dbe3cee1ce8a47fab35d8303c2950b1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.331496 8126 tablet_service.cc:1505] Processing CreateTablet for tablet d59acfb0e12643ebbb406dbda7bef29b (DEFAULT_TABLE table=default.Table_e4f1369ad6ad4110b5854544d4a266bd [id=ccea14281b2c420a99cede953a8de44d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.331696 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d59acfb0e12643ebbb406dbda7bef29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.331990 8394 tablet_service.cc:1505] Processing CreateTablet for tablet c0f428e5c6fe43fc8f78761d868ddd09 (DEFAULT_TABLE table=default.Table_a044870c006d4bde90c803732430cbe5 [id=0ee1306a57fd419497c97f844e58157e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.331990 8264 tablet_service.cc:1505] Processing CreateTablet for tablet c0f428e5c6fe43fc8f78761d868ddd09 (DEFAULT_TABLE table=default.Table_a044870c006d4bde90c803732430cbe5 [id=0ee1306a57fd419497c97f844e58157e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.332199 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f428e5c6fe43fc8f78761d868ddd09. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.333653 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 09d65eef81b2421c92678015b6da4fbb (DEFAULT_TABLE table=default.Table_d611f94943f44256ad3b9ca490f3282c [id=6d4246a1b1ea4448b8fbf6a5abecfed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.333925 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 874b6660d86b48ebad03ecc54588da28 (DEFAULT_TABLE table=default.Table_c7e0f5beff234f17b69504efc678924d [id=181b50c2d29a4b17b8189e3abb10a420]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.333998 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 7b552f63895449aea79cbd8867cc063c (DEFAULT_TABLE table=default.Table_8c2ee567a0b74836997f0d500266c256 [id=efaa3797242b4a238c7aac2d7427451c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.334172 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b552f63895449aea79cbd8867cc063c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.336206 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 7b552f63895449aea79cbd8867cc063c (DEFAULT_TABLE table=default.Table_8c2ee567a0b74836997f0d500266c256 [id=efaa3797242b4a238c7aac2d7427451c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.336531 8394 tablet_service.cc:1505] Processing CreateTablet for tablet b46554b0df894beb8141126aa7ee7b8e (DEFAULT_TABLE table=default.Table_c510beb7614e484c8b45ff008bca293c [id=9e5ad11a9ea34a3a8c629f48142b68f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.336694 8256 tablet_service.cc:1505] Processing CreateTablet for tablet b46554b0df894beb8141126aa7ee7b8e (DEFAULT_TABLE table=default.Table_c510beb7614e484c8b45ff008bca293c [id=9e5ad11a9ea34a3a8c629f48142b68f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.336782 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46554b0df894beb8141126aa7ee7b8e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:54.336747 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b60ad574ddda4f7d9c66bf24dd5c612a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.337292 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 81b521c9f57844bb9873f6b26b5cf661 (DEFAULT_TABLE table=default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.337944 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 81b521c9f57844bb9873f6b26b5cf661 (DEFAULT_TABLE table=default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.338071 8398 tablet_service.cc:1505] Processing CreateTablet for tablet f08a6e30211a4d59934c36c7a26ac983 (DEFAULT_TABLE table=default.Table_2de1f8c1f3564959a40963351d7d60b6 [id=b9c53ab5608f49309950ed79b8272710]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.338171 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b521c9f57844bb9873f6b26b5cf661. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.338163 8254 tablet_service.cc:1505] Processing CreateTablet for tablet f08a6e30211a4d59934c36c7a26ac983 (DEFAULT_TABLE table=default.Table_2de1f8c1f3564959a40963351d7d60b6 [id=b9c53ab5608f49309950ed79b8272710]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.338752 8253 tablet_service.cc:1505] Processing CreateTablet for tablet d59acfb0e12643ebbb406dbda7bef29b (DEFAULT_TABLE table=default.Table_e4f1369ad6ad4110b5854544d4a266bd [id=ccea14281b2c420a99cede953a8de44d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.339090 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f08a6e30211a4d59934c36c7a26ac983. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.339260 8397 tablet_service.cc:1505] Processing CreateTablet for tablet d59acfb0e12643ebbb406dbda7bef29b (DEFAULT_TABLE table=default.Table_e4f1369ad6ad4110b5854544d4a266bd [id=ccea14281b2c420a99cede953a8de44d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.339496 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d59acfb0e12643ebbb406dbda7bef29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.340049 8396 tablet_service.cc:1505] Processing CreateTablet for tablet f6ac7b5fff0848cebf710ba3ed50def4 (DEFAULT_TABLE table=default.Table_bd14742bd9f943cc93df997dfecd2719 [id=6f3e689a22ae4160ae7537d955c06a90]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.340257 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6ac7b5fff0848cebf710ba3ed50def4. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:54.341359 7953 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 03959a61996b40d8b9ef8b4b9708f017 (table default.Table_5da94c10f01846a59e4ce4ff530330d3 [id=bc83d2963bb74c2d9b914dee53bcf6ba], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 03959a61996b40d8b9ef8b4b9708f017 consensus state has no leader
I20251028 02:59:54.344919 8546 tablet_bootstrap.cc:654] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.344904 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 67c6ceefa3294af8b68150609f46c117 (table default.Table_f9254be0618c49f8a6a36fb9fc10354a [id=86c2f59df3dd4d2195d18835dceff79b]). Expected version 1 got 0
I20251028 02:59:54.345054 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 68ca4ff164b24b7498f953ec11cf46be (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:54.345139 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.346694 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_153d49d056a3458a9565c70d954187ae" } new_table_name: "default.Table_3fed043d81f54a3288c752ba83b12cf6" modify_external_catalogs: true
I20251028 02:59:54.350579 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet 309a9d10178e4984a18e10dfd11cba22 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.351989 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet a8331b21af69463cb7589619e784051d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.352195 8550 tablet_replica.cc:333] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.352340 8550 raft_consensus.cc:2243] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.352380 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 322cbd8253ce45118ad14e8f8f2f1f74 (DEFAULT_TABLE table=default.Table_610f127f3f9348309aa1ce4b6537e5b1 [id=40f8798a68d54ea991ef9bf6887dfe36]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.352448 8550 raft_consensus.cc:2272] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.352659 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 322cbd8253ce45118ad14e8f8f2f1f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.357856 8420 raft_consensus.cc:1275] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:54.358136 8283 raft_consensus.cc:1275] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:54.358198 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4bcdf0368e394d649086042f1fe9774b
I20251028 02:59:54.358446 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 322cbd8253ce45118ad14e8f8f2f1f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.358481 7965 catalog_manager.cc:6719] ReplaceTablet: tablet 4bcdf0368e394d649086042f1fe9774b deleted and replaced by tablet c21f794ec1284f5d99521954ed7ba9c9
I20251028 02:59:54.358668 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a3096fde2ac94ab197ce5e659c4d2be0
W20251028 02:59:54.359169 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b46554b0df894beb8141126aa7ee7b8e consensus state has no leader
I20251028 02:59:54.359478 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6ac7b5fff0848cebf710ba3ed50def4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.360828 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 04d271c95c2847ca8b1fd2d55741117c
I20251028 02:59:54.361470 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 04d271c95c2847ca8b1fd2d55741117c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.361713 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd50e7d4e28541de9799833c53fe7c02. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:54.362159 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 04d271c95c2847ca8b1fd2d55741117c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04d271c95c2847ca8b1fd2d55741117c already in progress: creating tablet
I20251028 02:59:54.362479 8544 tablet_bootstrap.cc:492] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.362603 8544 ts_tablet_manager.cc:1403] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.067s user 0.001s sys 0.000s
I20251028 02:59:54.363276 8544 raft_consensus.cc:359] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.363444 8544 raft_consensus.cc:385] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.363487 8544 raft_consensus.cc:740] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.363652 8544 consensus_queue.cc:260] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.363910 8264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f428e5c6fe43fc8f78761d868ddd09. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.364038 8544 ts_tablet_manager.cc:1434] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251028 02:59:54.364290 8544 tablet_bootstrap.cc:492] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.365355 8544 tablet_bootstrap.cc:654] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.366073 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet 4bcdf0368e394d649086042f1fe9774b with delete_type TABLET_DATA_DELETED (replaced by tablet c21f794ec1284f5d99521954ed7ba9c9) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.366318 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2ad18e95c3a14e039a63bebaa46268fa" } modify_external_catalogs: true
I20251028 02:59:54.366430 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2ad18e95c3a14e039a63bebaa46268fa" } modify_external_catalogs: true
I20251028 02:59:54.366598 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09d65eef81b2421c92678015b6da4fbb. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:54.367089 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4bcdf0368e394d649086042f1fe9774b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4bcdf0368e394d649086042f1fe9774b already in progress: creating tablet
I20251028 02:59:54.367713 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 874b6660d86b48ebad03ecc54588da28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.367939 8562 consensus_queue.cc:1048] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.368193 8562 consensus_queue.cc:1048] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.369856 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b552f63895449aea79cbd8867cc063c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.371470 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46554b0df894beb8141126aa7ee7b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.373574 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81b521c9f57844bb9873f6b26b5cf661. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.375530 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f08a6e30211a4d59934c36c7a26ac983. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:54.375684 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 529b865389f94471a6849d17ff206e4c consensus state has no leader
I20251028 02:59:54.377565 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d59acfb0e12643ebbb406dbda7bef29b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.379441 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dfc5240893546d7a6a5397cd08044dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.380890 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (table default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]). Expected version 1 got 0
I20251028 02:59:54.382018 8553 raft_consensus.cc:493] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.382107 8553 raft_consensus.cc:515] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.382553 8553 leader_election.cc:290] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.383059 9122 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 02:59:54.387466 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4ff8c974231f4b8aa18c18a05ad3e738" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:54.387626 8420 raft_consensus.cc:2468] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:54.387820 9048 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 02:59:54.391976 8220 leader_election.cc:304] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:54.391956 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_93edfbd4d0764b0aafb235f775be773f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.392228 8965 raft_consensus.cc:2804] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:54.392304 8965 raft_consensus.cc:493] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:54.392344 8965 raft_consensus.cc:3060] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:54.392402 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93edfbd4d0764b0aafb235f775be773f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.393285 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_44487777ef26464c9cbbb195199247a0" } modify_external_catalogs: true
I20251028 02:59:54.393384 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_44487777ef26464c9cbbb195199247a0" } modify_external_catalogs: true
I20251028 02:59:54.393648 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2ca85f32e093464c8bfade073ee1e7c3" } new_table_name: "default.Table_d39b99f0145b444a926eb0ecda4662e6" modify_external_catalogs: true
I20251028 02:59:54.396021 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4ff8c974231f4b8aa18c18a05ad3e738" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:54.396708 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ba35c8fe21f140cda8a4b1ad7568ec75
I20251028 02:59:54.399936 8965 raft_consensus.cc:515] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.400342 8965 leader_election.cc:290] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.400660 8616 raft_consensus.cc:493] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.400756 8616 raft_consensus.cc:515] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.400851 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4ff8c974231f4b8aa18c18a05ad3e738" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:54.400934 8417 raft_consensus.cc:3060] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.401172 8616 leader_election.cc:290] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.401669 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9860023c7b564342ba3145eafe352beb" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:54.401697 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bd1f7ca82cb84916aa7e1cbc5d217089"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.402024 8417 raft_consensus.cc:2468] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
W20251028 02:59:54.402148 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd1f7ca82cb84916aa7e1cbc5d217089 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:54.403014 8354 leader_election.cc:343] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:54.403509 8218 leader_election.cc:343] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.403909 8220 leader_election.cc:304] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:54.404170 8556 raft_consensus.cc:2804] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:54.404246 8556 raft_consensus.cc:697] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:54.404387 8556 consensus_queue.cc:237] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.408447 9023 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 02:59:54.410145 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_75e654d01f4147b3b7d928c99967c5cc" } modify_external_catalogs: true
I20251028 02:59:54.410250 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_75e654d01f4147b3b7d928c99967c5cc" } modify_external_catalogs: true
I20251028 02:59:54.410408 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 68ca4ff164b24b7498f953ec11cf46be (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:54.410526 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2b7f5657d1534447a3a29a01698581e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.410849 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b7f5657d1534447a3a29a01698581e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.412849 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 45572222af854b34a5f26a00f8ad9615 (DEFAULT_TABLE table=default.Table_65ab6aba20224249bcf1b91e6c9c65dd [id=94895dadf0e34a19a454e8755c61c785]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.413110 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45572222af854b34a5f26a00f8ad9615. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.415997 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4ff8c974231f4b8aa18c18a05ad3e738" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:54.417279 8254 tablet_service.cc:1505] Processing CreateTablet for tablet a19eeac727e142579e34a301a6e2db02 (DEFAULT_TABLE table=default.Table_635064f8d8484a978cd3e389004f2969 [id=effb15303ea6421a8eebaa32a124678e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.417773 8616 raft_consensus.cc:493] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.417892 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet ba35c8fe21f140cda8a4b1ad7568ec75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.417889 8616 raft_consensus.cc:515] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.418175 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 3f06af61617e4be9a753692e9ad4978d (DEFAULT_TABLE table=default.Table_a3c391d8f9a14324b6ae88bd860a0aa9 [id=7f9bfe24f9e24488bb02c120633dfcb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.418248 8616 leader_election.cc:290] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 02:59:54.418535 8218 leader_election.cc:343] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.420414 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 45572222af854b34a5f26a00f8ad9615 (DEFAULT_TABLE table=default.Table_65ab6aba20224249bcf1b91e6c9c65dd [id=94895dadf0e34a19a454e8755c61c785]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.420543 8125 tablet_service.cc:1505] Processing CreateTablet for tablet a19eeac727e142579e34a301a6e2db02 (DEFAULT_TABLE table=default.Table_635064f8d8484a978cd3e389004f2969 [id=effb15303ea6421a8eebaa32a124678e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.420686 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45572222af854b34a5f26a00f8ad9615. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.420791 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 04d271c95c2847ca8b1fd2d55741117c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.421815 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet ba35c8fe21f140cda8a4b1ad7568ec75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.422175 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a19eeac727e142579e34a301a6e2db02. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.420423 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 3f06af61617e4be9a753692e9ad4978d (DEFAULT_TABLE table=default.Table_a3c391d8f9a14324b6ae88bd860a0aa9 [id=7f9bfe24f9e24488bb02c120633dfcb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.423471 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f06af61617e4be9a753692e9ad4978d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.424136 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7528e7ff127c475789eda805467f0b41" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:54.424522 8354 leader_election.cc:343] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.424758 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a3c391d8f9a14324b6ae88bd860a0aa9" } new_table_name: "default.Table_1bcb25d3e7d34d09b41809bee9767f40" modify_external_catalogs: true
I20251028 02:59:54.426817 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 4bcdf0368e394d649086042f1fe9774b with delete_type TABLET_DATA_DELETED (replaced by tablet c21f794ec1284f5d99521954ed7ba9c9) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.427671 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 36d7cf962de845cfaca691be18a1bd63 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.427934 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 4bcdf0368e394d649086042f1fe9774b with delete_type TABLET_DATA_DELETED (replaced by tablet c21f794ec1284f5d99521954ed7ba9c9) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.428100 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet ba35c8fe21f140cda8a4b1ad7568ec75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.428066 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 3f06af61617e4be9a753692e9ad4978d (DEFAULT_TABLE table=default.Table_a3c391d8f9a14324b6ae88bd860a0aa9 [id=7f9bfe24f9e24488bb02c120633dfcb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.428369 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f06af61617e4be9a753692e9ad4978d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.428449 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet a3096fde2ac94ab197ce5e659c4d2be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.428659 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 04d271c95c2847ca8b1fd2d55741117c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.428699 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 45572222af854b34a5f26a00f8ad9615 (DEFAULT_TABLE table=default.Table_65ab6aba20224249bcf1b91e6c9c65dd [id=94895dadf0e34a19a454e8755c61c785]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.428951 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45572222af854b34a5f26a00f8ad9615. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.428941 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet a3096fde2ac94ab197ce5e659c4d2be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.430708 8401 tablet_service.cc:1505] Processing CreateTablet for tablet a19eeac727e142579e34a301a6e2db02 (DEFAULT_TABLE table=default.Table_635064f8d8484a978cd3e389004f2969 [id=effb15303ea6421a8eebaa32a124678e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.430938 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a19eeac727e142579e34a301a6e2db02. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.430912 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 1e9234b72f4448089fe7ff6ba49483b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.431224 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9860023c7b564342ba3145eafe352beb" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.431759 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7528e7ff127c475789eda805467f0b41" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:54.432092 8356 leader_election.cc:343] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.432166 8356 leader_election.cc:304] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:54.432490 8356 leader_election.cc:343] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.432548 8356 leader_election.cc:304] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.432724 8549 raft_consensus.cc:2749] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.432966 8549 raft_consensus.cc:2749] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251028 02:59:54.433223 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4bcdf0368e394d649086042f1fe9774b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4bcdf0368e394d649086042f1fe9774b already in progress: creating tablet
W20251028 02:59:54.433353 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ba35c8fe21f140cda8a4b1ad7568ec75 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba35c8fe21f140cda8a4b1ad7568ec75 already in progress: creating tablet
W20251028 02:59:54.433442 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a3096fde2ac94ab197ce5e659c4d2be0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3096fde2ac94ab197ce5e659c4d2be0 already in progress: creating tablet
W20251028 02:59:54.433550 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 04d271c95c2847ca8b1fd2d55741117c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04d271c95c2847ca8b1fd2d55741117c already in progress: creating tablet
I20251028 02:59:54.435703 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a19eeac727e142579e34a301a6e2db02. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.437258 7967 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_d802b3738006457a9d34cd51d2663e92 [id=a6ae2be5375e4328848f28f43dfc49ab] alter complete (version 5)
W20251028 02:59:54.437729 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3f06af61617e4be9a753692e9ad4978d consensus state has no leader
I20251028 02:59:54.438695 9122 tablet.cc:1722] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.440539 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 7528e7ff127c475789eda805467f0b41 with delete_type TABLET_DATA_DELETED (replaced by tablet 5428db004de74f7492c9b8999e53020e) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.441694 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 36d7cf962de845cfaca691be18a1bd63
I20251028 02:59:54.441977 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 36d7cf962de845cfaca691be18a1bd63 deleted and replaced by tablet 4902bc3398754802a5c571036cd19fa7
I20251028 02:59:54.442546 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet 36d7cf962de845cfaca691be18a1bd63 with delete_type TABLET_DATA_DELETED (replaced by tablet 4902bc3398754802a5c571036cd19fa7) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.440876 9190 consensus_queue.cc:1048] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.441277 9048 tablet.cc:1722] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.443624 8551 tablet_replica.cc:333] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.443765 8551 raft_consensus.cc:2243] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.443862 8552 tablet_replica.cc:333] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.443874 8551 raft_consensus.cc:2272] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.444000 8552 raft_consensus.cc:2243] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.444121 8552 raft_consensus.cc:2272] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.444415 8551 ts_tablet_manager.cc:1916] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.444470 8552 ts_tablet_manager.cc:1916] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.444878 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f06af61617e4be9a753692e9ad4978d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.446691 8552 ts_tablet_manager.cc:1929] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20251028 02:59:54.446751 8552 log.cc:1199] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/36d7cf962de845cfaca691be18a1bd63
I20251028 02:59:54.446753 8551 ts_tablet_manager.cc:1929] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.446818 8551 log.cc:1199] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/7528e7ff127c475789eda805467f0b41
I20251028 02:59:54.447013 8552 ts_tablet_manager.cc:1950] T 36d7cf962de845cfaca691be18a1bd63 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.447147 8551 ts_tablet_manager.cc:1950] T 7528e7ff127c475789eda805467f0b41 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 02:59:54.447889 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ba35c8fe21f140cda8a4b1ad7568ec75 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba35c8fe21f140cda8a4b1ad7568ec75 already in progress: creating tablet
W20251028 02:59:54.448060 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a3096fde2ac94ab197ce5e659c4d2be0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3096fde2ac94ab197ce5e659c4d2be0 already in progress: creating tablet
I20251028 02:59:54.448577 8649 mvcc.cc:204] Tried to move back new op lower bound from 7215597135280050176 to 7215597134154907648. Current Snapshot: MvccSnapshot[applied={T|T < 7215597135280050176}]
I20251028 02:59:54.448750 9023 tablet.cc:1722] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:54.442951 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 36d7cf962de845cfaca691be18a1bd63 with delete_type TABLET_DATA_DELETED (replaced by tablet 4902bc3398754802a5c571036cd19fa7) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.449569 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 7528e7ff127c475789eda805467f0b41 (table default.Table_663247205d844151bccc25218dcf39a5 [id=d40c0ee824ba423395e475f68276b5d7]) successfully deleted
I20251028 02:59:54.449796 8551 tablet_replica.cc:333] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.449950 8551 raft_consensus.cc:2243] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:54.450165 8551 raft_consensus.cc:2272] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.450572 8551 ts_tablet_manager.cc:1916] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.451339 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 68ca4ff164b24b7498f953ec11cf46be (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]). Expected version 2 got 0
I20251028 02:59:54.451354 9122 tablet.cc:1722] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:54.452925 8551 ts_tablet_manager.cc:1929] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20251028 02:59:54.452988 8551 log.cc:1199] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/36d7cf962de845cfaca691be18a1bd63
I20251028 02:59:54.454146 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 36d7cf962de845cfaca691be18a1bd63 (table default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]) successfully deleted
I20251028 02:59:54.454209 8551 ts_tablet_manager.cc:1950] T 36d7cf962de845cfaca691be18a1bd63 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.454828 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 36d7cf962de845cfaca691be18a1bd63 (table default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]) successfully deleted
I20251028 02:59:54.457553 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 529b865389f94471a6849d17ff206e4c from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.457717 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 37dcf605a2984be1817d9621e4a41448 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.460080 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 36d7cf962de845cfaca691be18a1bd63 with delete_type TABLET_DATA_DELETED (replaced by tablet 4902bc3398754802a5c571036cd19fa7) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.462740 9023 tablet.cc:1722] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.467023 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet a3096fde2ac94ab197ce5e659c4d2be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.467772 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 37dcf605a2984be1817d9621e4a41448
I20251028 02:59:54.468047 7967 catalog_manager.cc:6719] ReplaceTablet: tablet 37dcf605a2984be1817d9621e4a41448 deleted and replaced by tablet 1de3acf65f744368ab31dc1c82ac554f
I20251028 02:59:54.468529 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 37dcf605a2984be1817d9621e4a41448 with delete_type TABLET_DATA_DELETED (replaced by tablet 1de3acf65f744368ab31dc1c82ac554f) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.468592 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 37dcf605a2984be1817d9621e4a41448 with delete_type TABLET_DATA_DELETED (replaced by tablet 1de3acf65f744368ab31dc1c82ac554f) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.468766 8552 tablet_replica.cc:333] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.468791 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 37dcf605a2984be1817d9621e4a41448 with delete_type TABLET_DATA_DELETED (replaced by tablet 1de3acf65f744368ab31dc1c82ac554f) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.468921 8552 raft_consensus.cc:2243] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:54.469122 8552 raft_consensus.cc:2272] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.469727 8552 ts_tablet_manager.cc:1916] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.470654 8545 tablet_bootstrap.cc:492] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.470743 8545 ts_tablet_manager.cc:1403] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.170s user 0.002s sys 0.000s
I20251028 02:59:54.471359 8545 raft_consensus.cc:359] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.471464 8545 raft_consensus.cc:385] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.471495 8545 raft_consensus.cc:740] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.471669 8545 consensus_queue.cc:260] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.471971 8545 ts_tablet_manager.cc:1434] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.472278 8552 ts_tablet_manager.cc:1929] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:54.472344 8552 log.cc:1199] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/37dcf605a2984be1817d9621e4a41448
I20251028 02:59:54.472599 8552 ts_tablet_manager.cc:1950] T 37dcf605a2984be1817d9621e4a41448 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.473256 8545 tablet_bootstrap.cc:492] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.473723 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 37dcf605a2984be1817d9621e4a41448 (table default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc]) successfully deleted
I20251028 02:59:54.474014 8551 tablet_replica.cc:333] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.474064 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5a5158617c6246c29742d950e6c2f0cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.474504 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a5158617c6246c29742d950e6c2f0cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.474604 8551 raft_consensus.cc:2243] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.474731 8551 raft_consensus.cc:2272] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.475385 8545 tablet_bootstrap.cc:654] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.475656 8551 ts_tablet_manager.cc:1916] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.476128 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2ab414dbe9144aef8fd72a7f5a95577d
I20251028 02:59:54.476815 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ab414dbe9144aef8fd72a7f5a95577d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.476819 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ab414dbe9144aef8fd72a7f5a95577d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.476933 8550 ts_tablet_manager.cc:1916] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:54.479458 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 2ab414dbe9144aef8fd72a7f5a95577d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ab414dbe9144aef8fd72a7f5a95577d already in progress: creating tablet
I20251028 02:59:54.478767 8551 ts_tablet_manager.cc:1929] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:54.479641 8550 ts_tablet_manager.cc:1929] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.479679 8551 log.cc:1199] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/37dcf605a2984be1817d9621e4a41448
I20251028 02:59:54.479720 8550 log.cc:1199] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a8331b21af69463cb7589619e784051d
I20251028 02:59:54.479951 8550 ts_tablet_manager.cc:1950] T a8331b21af69463cb7589619e784051d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.479954 8551 ts_tablet_manager.cc:1950] T 37dcf605a2984be1817d9621e4a41448 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.480599 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 37dcf605a2984be1817d9621e4a41448 (table default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc]) successfully deleted
W20251028 02:59:54.481038 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 2ab414dbe9144aef8fd72a7f5a95577d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ab414dbe9144aef8fd72a7f5a95577d already in progress: creating tablet
I20251028 02:59:54.481410 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a8331b21af69463cb7589619e784051d (table default.Table_f73fbea5f42f4caa815469fbb7504f49 [id=79b638b293df431cbd580e62d24e8238]) successfully deleted
I20251028 02:59:54.481776 8550 tablet_replica.cc:333] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.481906 8550 raft_consensus.cc:2243] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.482004 8550 raft_consensus.cc:2272] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.482517 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ab414dbe9144aef8fd72a7f5a95577d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.483446 8550 ts_tablet_manager.cc:1916] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:54.483242 9177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20251028 02:59:54.483567 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 49b0b3459d434cf2810029ce205e745c
W20251028 02:59:54.483603 9177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251028 02:59:54.483680 9177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251028 02:59:54.484346 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 49b0b3459d434cf2810029ce205e745c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.484496 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 49b0b3459d434cf2810029ce205e745c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:54.484942 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 49b0b3459d434cf2810029ce205e745c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 49b0b3459d434cf2810029ce205e745c already in progress: creating tablet
W20251028 02:59:54.485462 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 49b0b3459d434cf2810029ce205e745c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 49b0b3459d434cf2810029ce205e745c already in progress: creating tablet
I20251028 02:59:54.485621 8550 ts_tablet_manager.cc:1929] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.485682 8550 log.cc:1199] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/1e9234b72f4448089fe7ff6ba49483b7
I20251028 02:59:54.485880 8550 ts_tablet_manager.cc:1950] T 1e9234b72f4448089fe7ff6ba49483b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.486384 8562 raft_consensus.cc:493] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.486421 8545 tablet_bootstrap.cc:492] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.486454 8562 raft_consensus.cc:515] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.486500 8545 ts_tablet_manager.cc:1403] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20251028 02:59:54.486773 8562 leader_election.cc:290] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.486950 8545 raft_consensus.cc:359] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.487043 8545 raft_consensus.cc:385] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.487071 8545 raft_consensus.cc:740] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.487212 8545 consensus_queue.cc:260] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.487360 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7f804f81d70749b5bdefeca4e0bad6fe" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:54.487361 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7f804f81d70749b5bdefeca4e0bad6fe" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.487617 8545 ts_tablet_manager.cc:1434] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:54.487783 8545 tablet_bootstrap.cc:492] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.488729 8545 tablet_bootstrap.cc:654] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.488871 8083 leader_election.cc:304] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:54.489084 8562 raft_consensus.cc:2749] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.490473 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 49b0b3459d434cf2810029ce205e745c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.491843 8546 tablet_bootstrap.cc:492] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.491946 8546 ts_tablet_manager.cc:1403] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.185s user 0.000s sys 0.002s
I20251028 02:59:54.492235 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_dca026923a844b5faf18d96820d52dbd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.492436 8546 raft_consensus.cc:359] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.492528 8546 raft_consensus.cc:385] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.492556 8546 raft_consensus.cc:740] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
W20251028 02:59:54.492686 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dca026923a844b5faf18d96820d52dbd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.492668 8546 consensus_queue.cc:260] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.492980 8546 ts_tablet_manager.cc:1434] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:54.493171 8546 tablet_bootstrap.cc:492] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.494155 8546 tablet_bootstrap.cc:654] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.495092 8544 tablet_bootstrap.cc:492] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.495180 8544 ts_tablet_manager.cc:1403] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.131s user 0.001s sys 0.000s
I20251028 02:59:54.495684 8545 tablet_bootstrap.cc:492] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.495698 8544 raft_consensus.cc:359] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.495767 8545 ts_tablet_manager.cc:1403] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20251028 02:59:54.495807 8544 raft_consensus.cc:385] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.495842 8544 raft_consensus.cc:740] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
W20251028 02:59:54.495944 9177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251028 02:59:54.495997 9177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20251028 02:59:54.495957 8544 consensus_queue.cc:260] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:54.496037 9177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251028 02:59:54.496066 9177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251028 02:59:54.496224 8544 ts_tablet_manager.cc:1434] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.496255 8545 raft_consensus.cc:359] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.496340 8545 raft_consensus.cc:385] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.496364 8545 raft_consensus.cc:740] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.496378 8544 tablet_bootstrap.cc:492] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.496488 8545 consensus_queue.cc:260] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.497334 8544 tablet_bootstrap.cc:654] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.498418 8545 ts_tablet_manager.cc:1434] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:54.500461 8147 raft_consensus.cc:3060] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.501029 8545 tablet_bootstrap.cc:492] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.502197 8938 consensus_queue.cc:1048] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.502483 9177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.154.148:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.189:45473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.154.189
--webserver_port=36469
--never_fsync=true
--heap_profile_path=/tmp/kudu.9177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 60f5e5267b92c39485a66121d3ce3cc7ef57b0e0
build type DEBUG
built by None at 28 Oct 2025 02:43:12 UTC on 24a791456cd2
build id 8750
I20251028 02:59:54.502707 7965 catalog_manager.cc:5649] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:54.502972 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]). Expected version 1 got 0
I20251028 02:59:54.503093 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_0e8ca626159c41aeb8e360f6e038f7a3 [id=eb0e103661fc4bc7b27bb78fc333fd56]). Expected version 1 got 0
I20251028 02:59:54.503619 7966 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_d39b99f0145b444a926eb0ecda4662e6 [id=a6d01f64a2e34c5781f1e090b1d2cd88] alter complete (version 1)
I20251028 02:59:54.505190 8545 tablet_bootstrap.cc:654] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.505528 9177 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251028 02:59:54.506412 8545 tablet_bootstrap.cc:492] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.506496 8545 ts_tablet_manager.cc:1403] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:54.506449 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_efbcb24780904b469cfe628420a6addd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.506785 9048 tablet.cc:1722] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20251028 02:59:54.506858 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efbcb24780904b469cfe628420a6addd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.506958 8545 raft_consensus.cc:359] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.507051 8545 raft_consensus.cc:385] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.507084 8545 raft_consensus.cc:740] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.507197 8545 consensus_queue.cc:260] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.507475 8545 ts_tablet_manager.cc:1434] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:54.507647 8545 tablet_bootstrap.cc:492] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:54.507998 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 04d271c95c2847ca8b1fd2d55741117c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04d271c95c2847ca8b1fd2d55741117c already in progress: creating tablet
W20251028 02:59:54.508184 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ba35c8fe21f140cda8a4b1ad7568ec75 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba35c8fe21f140cda8a4b1ad7568ec75 already in progress: creating tablet
I20251028 02:59:54.508199 9177 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251028 02:59:54.508277 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4bcdf0368e394d649086042f1fe9774b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4bcdf0368e394d649086042f1fe9774b already in progress: creating tablet
I20251028 02:59:54.508594 8545 tablet_bootstrap.cc:654] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.511068 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 1e9234b72f4448089fe7ff6ba49483b7 (table default.Table_3d8776d793a54bd795fb38bd45442f51 [id=be1f381c3ca048d891f5d175fa72fc65]) successfully deleted
I20251028 02:59:54.511353 8550 tablet_replica.cc:333] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.511498 8550 raft_consensus.cc:2243] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.511620 8550 raft_consensus.cc:2272] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.512261 8550 ts_tablet_manager.cc:1916] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:54.516049 9254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:54.516480 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_0e8ca626159c41aeb8e360f6e038f7a3 [id=eb0e103661fc4bc7b27bb78fc333fd56]). Expected version 1 got 0
I20251028 02:59:54.516952 8550 ts_tablet_manager.cc:1929] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
W20251028 02:59:54.516968 9251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:54.517020 8550 log.cc:1199] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/36d7cf962de845cfaca691be18a1bd63
I20251028 02:59:54.517340 8550 ts_tablet_manager.cc:1950] T 36d7cf962de845cfaca691be18a1bd63 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.517833 9177 server_base.cc:1047] running on GCE node
W20251028 02:59:54.518818 9252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 02:59:54.519496 9177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251028 02:59:54.519706 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 56c8969da9dc464d9b463205e3a27965 (table default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]). Expected version 1 got 0
I20251028 02:59:54.519850 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_0e8ca626159c41aeb8e360f6e038f7a3 [id=eb0e103661fc4bc7b27bb78fc333fd56]). Expected version 1 got 0
I20251028 02:59:54.520732 9177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251028 02:59:54.521342 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet cf90b539dc8f47b0947d4e2ca859668a with delete_type TABLET_DATA_DELETED (replaced by tablet 4246c4bd2abf4569987ca346996445d4) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.521324 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet c083ca984a7347a6b5bb0f3a949a5f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.521507 8551 tablet_replica.cc:333] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.521639 8551 raft_consensus.cc:2243] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.521741 8551 raft_consensus.cc:2272] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.521842 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 36d7cf962de845cfaca691be18a1bd63 (table default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]) successfully deleted
I20251028 02:59:54.522358 8551 ts_tablet_manager.cc:1916] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.522534 8545 tablet_bootstrap.cc:492] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.522608 8545 ts_tablet_manager.cc:1403] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.001s
I20251028 02:59:54.523016 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet a3096fde2ac94ab197ce5e659c4d2be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.523219 8545 raft_consensus.cc:359] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.523355 8545 raft_consensus.cc:385] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.523389 8545 raft_consensus.cc:740] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.523522 8545 consensus_queue.cc:260] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.523838 8545 ts_tablet_manager.cc:1434] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251028 02:59:54.524072 8545 tablet_bootstrap.cc:492] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.524540 8546 tablet_bootstrap.cc:492] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.524629 8546 ts_tablet_manager.cc:1403] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.001s
I20251028 02:59:54.524963 8551 ts_tablet_manager.cc:1929] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.525029 8551 log.cc:1199] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c083ca984a7347a6b5bb0f3a949a5f75
I20251028 02:59:54.525235 8551 ts_tablet_manager.cc:1950] T c083ca984a7347a6b5bb0f3a949a5f75 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.525183 8546 raft_consensus.cc:359] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.525290 8546 raft_consensus.cc:385] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.525334 8546 raft_consensus.cc:740] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.525462 8546 consensus_queue.cc:260] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.525767 8546 ts_tablet_manager.cc:1434] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.525966 8546 tablet_bootstrap.cc:492] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.526070 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet ba35c8fe21f140cda8a4b1ad7568ec75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.526366 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 920623f3747046d9be594241e8c13733 (DEFAULT_TABLE table=default.Table_b60ad574ddda4f7d9c66bf24dd5c612a [id=bbd58ecf5bc44d18a12db6bb204c9c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.526646 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 920623f3747046d9be594241e8c13733. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.526950 8546 tablet_bootstrap.cc:654] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.527581 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 920623f3747046d9be594241e8c13733 (DEFAULT_TABLE table=default.Table_b60ad574ddda4f7d9c66bf24dd5c612a [id=bbd58ecf5bc44d18a12db6bb204c9c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.528323 8544 tablet_bootstrap.cc:492] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.528403 8544 ts_tablet_manager.cc:1403] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20251028 02:59:54.528829 8544 raft_consensus.cc:359] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.528924 8544 raft_consensus.cc:385] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.528957 8544 raft_consensus.cc:740] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.529085 8544 consensus_queue.cc:260] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.529357 8544 ts_tablet_manager.cc:1434] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.529534 8544 tablet_bootstrap.cc:492] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.530619 8544 tablet_bootstrap.cc:654] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.530975 8545 tablet_bootstrap.cc:654] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.532038 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet d107a510c8664999901320581e885ba9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a2ebe28dbd2439e963319ec6e515635) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.532341 8545 tablet_bootstrap.cc:492] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.532424 8545 ts_tablet_manager.cc:1403] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20251028 02:59:54.533250 9177 hybrid_clock.cc:648] HybridClock initialized: now 1761620394533155 us; error 104 us; skew 500 ppm
I20251028 02:59:54.533285 8553 raft_consensus.cc:493] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.533393 8553 raft_consensus.cc:515] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.533877 8553 leader_election.cc:290] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.534444 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "67c6ceefa3294af8b68150609f46c117" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:54.534596 8145 raft_consensus.cc:2468] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:54.534332 8545 raft_consensus.cc:359] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.534854 8545 raft_consensus.cc:385] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.534930 8545 raft_consensus.cc:740] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.535135 8545 consensus_queue.cc:260] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.535313 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "67c6ceefa3294af8b68150609f46c117" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:54.535352 8218 leader_election.cc:304] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:54.535431 8421 raft_consensus.cc:2468] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:54.536188 8553 raft_consensus.cc:2804] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:54.536065 7968 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_0e8ca626159c41aeb8e360f6e038f7a3 [id=eb0e103661fc4bc7b27bb78fc333fd56]). Expected version 1 got 0
I20251028 02:59:54.536252 8553 raft_consensus.cc:493] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:54.536298 8553 raft_consensus.cc:3060] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.536302 9177 webserver.cc:492] Webserver started at http://127.7.154.189:36469/ using document root <none> and password file <none>
I20251028 02:59:54.536718 7968 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]). Expected version 1 got 0
I20251028 02:59:54.536974 8545 ts_tablet_manager.cc:1434] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20251028 02:59:54.537113 9177 fs_manager.cc:362] Metadata directory not provided
I20251028 02:59:54.537200 9177 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 02:59:54.537379 8553 raft_consensus.cc:515] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.537757 8553 leader_election.cc:290] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.538077 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "67c6ceefa3294af8b68150609f46c117" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:54.538183 8421 raft_consensus.cc:3060] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.538218 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "67c6ceefa3294af8b68150609f46c117" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:54.538332 8145 raft_consensus.cc:3060] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.539156 8421 raft_consensus.cc:2468] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:54.539531 8220 leader_election.cc:304] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:54.539577 8544 tablet_bootstrap.cc:492] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.539664 8544 ts_tablet_manager.cc:1403] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.001s
I20251028 02:59:54.540177 8544 raft_consensus.cc:359] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.540302 8544 raft_consensus.cc:385] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.540300 8553 raft_consensus.cc:2804] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:54.540339 8544 raft_consensus.cc:740] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.540385 8553 raft_consensus.cc:697] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:54.540486 8544 consensus_queue.cc:260] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.540511 8553 consensus_queue.cc:237] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.540184 8145 raft_consensus.cc:2468] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:54.542953 9177 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20251028 02:59:54.543154 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fd9cc1847ff4df7a09e2e698786d515 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.543385 8552 tablet_replica.cc:333] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.543529 8552 raft_consensus.cc:2243] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.543630 8552 raft_consensus.cc:2272] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.544127 8544 ts_tablet_manager.cc:1434] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:54.544368 8552 ts_tablet_manager.cc:1916] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.545143 8551 tablet_replica.cc:333] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.545274 8551 raft_consensus.cc:2243] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.545365 8551 raft_consensus.cc:2272] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.545625 8545 tablet_bootstrap.cc:492] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.545838 8544 tablet_bootstrap.cc:492] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.546752 8545 tablet_bootstrap.cc:654] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.546854 8552 ts_tablet_manager.cc:1929] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.546921 8552 log.cc:1199] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/5fd9cc1847ff4df7a09e2e698786d515
I20251028 02:59:54.547115 8544 tablet_bootstrap.cc:654] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.547143 8552 ts_tablet_manager.cc:1950] T 5fd9cc1847ff4df7a09e2e698786d515 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.547824 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 1094f4d2872449599e3eaf286def4f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.548076 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 5fd9cc1847ff4df7a09e2e698786d515 (table default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]) successfully deleted
I20251028 02:59:54.548280 8550 tablet_replica.cc:333] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.548396 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet c083ca984a7347a6b5bb0f3a949a5f75 (table default.Table_d1b772f7ad114598bf453e2265c5e436 [id=73579e6e9da94fa699b3cb5bb5cec89e]) successfully deleted
I20251028 02:59:54.548570 8550 raft_consensus.cc:2243] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.548738 8550 raft_consensus.cc:2272] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.548758 8551 ts_tablet_manager.cc:1916] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.549320 8550 ts_tablet_manager.cc:1916] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.551095 8551 ts_tablet_manager.cc:1929] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.551098 9266 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 02:59:54.551177 8551 log.cc:1199] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a3096fde2ac94ab197ce5e659c4d2be0
I20251028 02:59:54.552111 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_27f606e115954fd59700c86dabe70cae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.552217 8545 tablet_bootstrap.cc:492] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.552178 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3a29fd732cf24fcdadc69d00d2175216"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.552289 8545 ts_tablet_manager.cc:1403] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
W20251028 02:59:54.552561 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27f606e115954fd59700c86dabe70cae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:54.552568 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a29fd732cf24fcdadc69d00d2175216 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.552867 9177 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:54.552822 8545 raft_consensus.cc:359] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.552922 8545 raft_consensus.cc:385] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.552968 8545 raft_consensus.cc:740] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.553107 8545 consensus_queue.cc:260] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.552978 9177 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data,/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
uuid: "aac264d6d2274d3283bd3251ab77e583"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
I20251028 02:59:54.553571 8545 ts_tablet_manager.cc:1434] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.553519 9177 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 02:59:54.554136 8544 tablet_bootstrap.cc:492] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.554276 8544 ts_tablet_manager.cc:1403] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:54.554811 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_41a9163bd20143ff921de1c3d023fcfd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.554937 8544 raft_consensus.cc:359] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.555056 8544 raft_consensus.cc:385] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.555097 8544 raft_consensus.cc:740] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.555253 8544 consensus_queue.cc:260] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:54.555584 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41a9163bd20143ff921de1c3d023fcfd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.555842 8545 tablet_bootstrap.cc:492] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.556859 8551 ts_tablet_manager.cc:1950] T a3096fde2ac94ab197ce5e659c4d2be0 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.557471 8544 ts_tablet_manager.cc:1434] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:54.558192 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fd9cc1847ff4df7a09e2e698786d515 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.559451 8544 tablet_bootstrap.cc:492] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.560698 8545 tablet_bootstrap.cc:654] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.561640 8544 tablet_bootstrap.cc:654] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.562920 8544 tablet_bootstrap.cc:492] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.563014 8544 ts_tablet_manager.cc:1403] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:54.563604 8544 raft_consensus.cc:359] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.563726 8544 raft_consensus.cc:385] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.563763 8544 raft_consensus.cc:740] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.563898 8544 consensus_queue.cc:260] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.564250 8544 ts_tablet_manager.cc:1434] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.564471 8544 tablet_bootstrap.cc:492] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.565675 8544 tablet_bootstrap.cc:654] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.566859 8545 tablet_bootstrap.cc:492] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.566952 8545 ts_tablet_manager.cc:1403] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:54.567909 8562 raft_consensus.cc:493] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.568003 8562 raft_consensus.cc:515] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.568423 8562 leader_election.cc:290] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.568869 8545 raft_consensus.cc:359] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.568979 8545 raft_consensus.cc:385] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.569010 8545 raft_consensus.cc:740] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.569141 8545 consensus_queue.cc:260] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.569330 8549 raft_consensus.cc:493] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.569392 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50e6d41d1cf746aab2802b5c6b2d58b3" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.569422 8549 raft_consensus.cc:515] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.569517 8284 raft_consensus.cc:2468] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:54.569847 8549 leader_election.cc:290] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.570259 8083 leader_election.cc:304] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:54.570644 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bc089d8155d4463082112a190f9da5b8" } new_table_name: "default.Table_a00976e88f0147c5b01560dba597ea82" modify_external_catalogs: true
I20251028 02:59:54.570840 8562 raft_consensus.cc:2804] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:54.570900 8562 raft_consensus.cc:493] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:54.570947 8562 raft_consensus.cc:3060] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.571971 8562 raft_consensus.cc:515] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.572237 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fc88d863fb634dc59631a752b9afe967" } modify_external_catalogs: true
I20251028 02:59:54.572331 8562 leader_election.cc:290] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.572342 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fc88d863fb634dc59631a752b9afe967" } modify_external_catalogs: true
I20251028 02:59:54.573055 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50e6d41d1cf746aab2802b5c6b2d58b3" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:54.573181 8283 raft_consensus.cc:3060] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.573165 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1b0dd83e47164c4da9d0b2311dfffc81" } modify_external_catalogs: true
I20251028 02:59:54.573252 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1b0dd83e47164c4da9d0b2311dfffc81" } modify_external_catalogs: true
I20251028 02:59:54.574184 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_635064f8d8484a978cd3e389004f2969" } new_table_name: "default.Table_0ac514a2be6a4e1b904149fd8fdc8eb3" modify_external_catalogs: true
I20251028 02:59:54.574301 8283 raft_consensus.cc:2468] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:54.574843 8083 leader_election.cc:304] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
W20251028 02:59:54.575356 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a3096fde2ac94ab197ce5e659c4d2be0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3096fde2ac94ab197ce5e659c4d2be0 already in progress: creating tablet
I20251028 02:59:54.575470 8545 ts_tablet_manager.cc:1434] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20251028 02:59:54.576476 8562 raft_consensus.cc:2804] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:54.576570 8562 raft_consensus.cc:697] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:54.576721 8562 consensus_queue.cc:237] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.577906 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]). Expected version 1 got 0
I20251028 02:59:54.579581 8544 tablet_bootstrap.cc:492] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.579684 8544 ts_tablet_manager.cc:1403] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.001s
I20251028 02:59:54.580169 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6f9b707ec0345249c3efec1c24a956e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:54.580216 8544 raft_consensus.cc:359] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.580320 8544 raft_consensus.cc:385] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.580349 8544 raft_consensus.cc:740] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.580456 8544 consensus_queue.cc:260] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.580709 8544 ts_tablet_manager.cc:1434] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:54.580881 8544 tablet_bootstrap.cc:492] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.582015 8544 tablet_bootstrap.cc:654] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.583359 9177 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 02:59:54.584100 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet d81b3fe6073d4380882057eb97907fbb from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.584265 9177 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251028 02:59:54.584527 9177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 02:59:54.584707 8544 tablet_bootstrap.cc:492] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.584805 8544 ts_tablet_manager.cc:1403] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:54.584918 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50e6d41d1cf746aab2802b5c6b2d58b3" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:54.585084 8417 raft_consensus.cc:3060] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.585279 8544 raft_consensus.cc:359] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.585350 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50e6d41d1cf746aab2802b5c6b2d58b3" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:54.585389 8544 raft_consensus.cc:385] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.585422 8544 raft_consensus.cc:740] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.586273 8417 raft_consensus.cc:2468] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:54.586915 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_afba81072ed644d4aac9d04bc5902e63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.587000 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a3096fde2ac94ab197ce5e659c4d2be0 (table default.Table_6d8f283a536e46fcbf3d8abcf9d703bb [id=a6d84958fd654087940c9806e2e3fbc0]) successfully deleted
I20251028 02:59:54.585520 8544 consensus_queue.cc:260] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:54.587360 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_afba81072ed644d4aac9d04bc5902e63 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.588250 8544 ts_tablet_manager.cc:1434] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251028 02:59:54.589001 8544 tablet_bootstrap.cc:492] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.590091 8544 tablet_bootstrap.cc:654] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.590885 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e6f9b707ec0345249c3efec1c24a956e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:54.591389 8356 leader_election.cc:343] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.591779 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 920623f3747046d9be594241e8c13733. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.591958 8544 tablet_bootstrap.cc:492] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.592031 8544 ts_tablet_manager.cc:1403] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251028 02:59:54.592446 8544 raft_consensus.cc:359] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.592554 8544 raft_consensus.cc:385] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.592582 8544 raft_consensus.cc:740] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.592675 8544 consensus_queue.cc:260] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.592921 8544 ts_tablet_manager.cc:1434] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.593118 8544 tablet_bootstrap.cc:492] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 02:59:54.593271 8354 leader_election.cc:343] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.593336 8354 leader_election.cc:304] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.591272 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 529b865389f94471a6849d17ff206e4c
I20251028 02:59:54.594105 8544 tablet_bootstrap.cc:654] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.594342 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 529b865389f94471a6849d17ff206e4c deleted and replaced by tablet 240b7da6f46c4c5ab8087ab1dd6275c5
I20251028 02:59:54.595278 7969 catalog_manager.cc:5649] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:54.595504 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 67c6ceefa3294af8b68150609f46c117 (table default.Table_f9254be0618c49f8a6a36fb9fc10354a [id=86c2f59df3dd4d2195d18835dceff79b]). Expected version 1 got 0
I20251028 02:59:54.595870 8549 raft_consensus.cc:2749] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.597417 8544 tablet_bootstrap.cc:492] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.597509 8544 ts_tablet_manager.cc:1403] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:54.597961 8544 raft_consensus.cc:359] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.598089 8544 raft_consensus.cc:385] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.598126 8544 raft_consensus.cc:740] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.598243 8544 consensus_queue.cc:260] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.598548 8544 ts_tablet_manager.cc:1434] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.598727 8544 tablet_bootstrap.cc:492] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.599977 8544 tablet_bootstrap.cc:654] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.600101 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 529b865389f94471a6849d17ff206e4c with delete_type TABLET_DATA_DELETED (replaced by tablet 240b7da6f46c4c5ab8087ab1dd6275c5) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.600454 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 920623f3747046d9be594241e8c13733 (DEFAULT_TABLE table=default.Table_b60ad574ddda4f7d9c66bf24dd5c612a [id=bbd58ecf5bc44d18a12db6bb204c9c84]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.600750 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 920623f3747046d9be594241e8c13733. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.600818 8126 tablet_service.cc:1505] Processing CreateTablet for tablet aa0776d1a8ef43449309221b86ed3923 (DEFAULT_TABLE table=default.Table_4457c9f159ad428c99a082b56e95f8ad [id=4570d7ddf33c447fb40d6411ea47e52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.601050 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa0776d1a8ef43449309221b86ed3923. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.601315 8399 tablet_service.cc:1505] Processing CreateTablet for tablet aa0776d1a8ef43449309221b86ed3923 (DEFAULT_TABLE table=default.Table_4457c9f159ad428c99a082b56e95f8ad [id=4570d7ddf33c447fb40d6411ea47e52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.601858 8546 tablet_bootstrap.cc:492] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.602008 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa0776d1a8ef43449309221b86ed3923. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.602316 8546 ts_tablet_manager.cc:1403] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.076s user 0.000s sys 0.002s
I20251028 02:59:54.602870 8544 tablet_bootstrap.cc:492] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.602941 8544 ts_tablet_manager.cc:1403] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:54.603116 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 3b43da9580244346ae512a7bc9b506d3 (DEFAULT_TABLE table=default.Table_bc089d8155d4463082112a190f9da5b8 [id=7244d96a49d34411bb8f9f7416904ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.603116 8258 tablet_service.cc:1505] Processing CreateTablet for tablet c21f794ec1284f5d99521954ed7ba9c9 (DEFAULT_TABLE table=default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.603354 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b43da9580244346ae512a7bc9b506d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.603377 8544 raft_consensus.cc:359] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.603480 8544 raft_consensus.cc:385] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.603510 8544 raft_consensus.cc:740] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.603622 8544 consensus_queue.cc:260] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.603893 8544 ts_tablet_manager.cc:1434] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.604084 8544 tablet_bootstrap.cc:492] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.605060 8544 tablet_bootstrap.cc:654] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.605341 8546 raft_consensus.cc:359] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.605357 8401 tablet_service.cc:1505] Processing CreateTablet for tablet c21f794ec1284f5d99521954ed7ba9c9 (DEFAULT_TABLE table=default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.605455 8546 raft_consensus.cc:385] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.605487 8546 raft_consensus.cc:740] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.605623 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c21f794ec1284f5d99521954ed7ba9c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.605623 8546 consensus_queue.cc:260] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.605940 8546 ts_tablet_manager.cc:1434] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:54.606091 8253 tablet_service.cc:1505] Processing CreateTablet for tablet 93ef23900c0e4498934208352eb706f8 (DEFAULT_TABLE table=default.Table_00a06d6bfdff457ab18a0404c944f780 [id=27cec05ebbf74601b28ed49f36637dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.606176 8546 tablet_bootstrap.cc:492] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.606432 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 3b43da9580244346ae512a7bc9b506d3 (DEFAULT_TABLE table=default.Table_bc089d8155d4463082112a190f9da5b8 [id=7244d96a49d34411bb8f9f7416904ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.606892 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 529b865389f94471a6849d17ff206e4c with delete_type TABLET_DATA_DELETED (replaced by tablet 240b7da6f46c4c5ab8087ab1dd6275c5) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.607141 8127 tablet_service.cc:1505] Processing CreateTablet for tablet c21f794ec1284f5d99521954ed7ba9c9 (DEFAULT_TABLE table=default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.607414 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c21f794ec1284f5d99521954ed7ba9c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.607899 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 93ef23900c0e4498934208352eb706f8 (DEFAULT_TABLE table=default.Table_00a06d6bfdff457ab18a0404c944f780 [id=27cec05ebbf74601b28ed49f36637dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.608126 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ef23900c0e4498934208352eb706f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.608306 8550 ts_tablet_manager.cc:1929] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:54.608376 8550 log.cc:1199] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/37dcf605a2984be1817d9621e4a41448
I20251028 02:59:54.608644 8550 ts_tablet_manager.cc:1950] T 37dcf605a2984be1817d9621e4a41448 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.609467 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 93ef23900c0e4498934208352eb706f8 (DEFAULT_TABLE table=default.Table_00a06d6bfdff457ab18a0404c944f780 [id=27cec05ebbf74601b28ed49f36637dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.609685 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ef23900c0e4498934208352eb706f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.609879 8550 tablet_replica.cc:333] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
W20251028 02:59:54.613143 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 529b865389f94471a6849d17ff206e4c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 529b865389f94471a6849d17ff206e4c already in progress: creating tablet
I20251028 02:59:54.613633 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 37dcf605a2984be1817d9621e4a41448 (table default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc]) successfully deleted
W20251028 02:59:54.613899 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 2ab414dbe9144aef8fd72a7f5a95577d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ab414dbe9144aef8fd72a7f5a95577d already in progress: creating tablet
W20251028 02:59:54.614117 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 49b0b3459d434cf2810029ce205e745c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 49b0b3459d434cf2810029ce205e745c already in progress: creating tablet
I20251028 02:59:54.605957 8260 tablet_service.cc:1505] Processing CreateTablet for tablet aa0776d1a8ef43449309221b86ed3923 (DEFAULT_TABLE table=default.Table_4457c9f159ad428c99a082b56e95f8ad [id=4570d7ddf33c447fb40d6411ea47e52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.615250 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 3b43da9580244346ae512a7bc9b506d3 (DEFAULT_TABLE table=default.Table_bc089d8155d4463082112a190f9da5b8 [id=7244d96a49d34411bb8f9f7416904ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.618830 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b43da9580244346ae512a7bc9b506d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.620222 9177 rpc_server.cc:307] RPC server started. Bound to: 127.7.154.189:45473
I20251028 02:59:54.621130 9327 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.154.189:45473 every 8 connection(s)
I20251028 02:59:54.621520 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c21f794ec1284f5d99521954ed7ba9c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.621613 9177 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-1/data/info.pb
I20251028 02:59:54.622630 8546 tablet_bootstrap.cc:654] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.623670 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ef23900c0e4498934208352eb706f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.612861 8562 raft_consensus.cc:493] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.624553 8562 raft_consensus.cc:515] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.625008 8562 leader_election.cc:290] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:54.626250 8083 leader_election.cc:343] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 5fd9cc1847ff4df7a09e2e698786d515
W20251028 02:59:54.626375 8083 leader_election.cc:343] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: 5fd9cc1847ff4df7a09e2e698786d515
I20251028 02:59:54.626421 8083 leader_election.cc:304] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:54.626617 8562 raft_consensus.cc:2749] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.610026 8550 raft_consensus.cc:2243] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.626837 8550 raft_consensus.cc:2272] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.627497 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b43da9580244346ae512a7bc9b506d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.628496 9328 sys_catalog.cc:263] Verifying existing consensus state
I20251028 02:59:54.628573 8550 ts_tablet_manager.cc:1916] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.629782 7786 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu as pid 9177
I20251028 02:59:54.630266 8546 tablet_bootstrap.cc:492] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.630357 8546 ts_tablet_manager.cc:1403] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.001s
I20251028 02:59:54.630451 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa0776d1a8ef43449309221b86ed3923. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.630688 9328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Bootstrap starting.
I20251028 02:59:54.630889 8546 raft_consensus.cc:359] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.631013 8546 raft_consensus.cc:385] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.631060 8546 raft_consensus.cc:740] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.631191 8546 consensus_queue.cc:260] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.631481 8546 ts_tablet_manager.cc:1434] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:54.631657 8546 tablet_bootstrap.cc:492] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.632589 8546 tablet_bootstrap.cc:654] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.636024 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 529b865389f94471a6849d17ff206e4c with delete_type TABLET_DATA_DELETED (replaced by tablet 240b7da6f46c4c5ab8087ab1dd6275c5) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.636390 8545 tablet_bootstrap.cc:492] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.637418 8545 tablet_bootstrap.cc:654] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.637992 8551 tablet_replica.cc:333] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.638432 8551 raft_consensus.cc:2243] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.638516 8562 raft_consensus.cc:493] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.638608 8551 raft_consensus.cc:2272] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.638624 8562 raft_consensus.cc:515] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.638978 8562 leader_election.cc:290] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.639678 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56c8969da9dc464d9b463205e3a27965" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:54.639818 8417 raft_consensus.cc:2468] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:54.640121 8083 leader_election.cc:304] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:54.640376 8562 raft_consensus.cc:2804] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:54.640422 8562 raft_consensus.cc:493] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:54.640455 8562 raft_consensus.cc:3060] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.641372 8562 raft_consensus.cc:515] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.641693 8562 leader_election.cc:290] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.642251 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56c8969da9dc464d9b463205e3a27965" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 02:59:54.642637 8083 leader_election.cc:343] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.642987 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56c8969da9dc464d9b463205e3a27965" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:54.643115 8417 raft_consensus.cc:3060] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.643373 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56c8969da9dc464d9b463205e3a27965" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:54.643684 8083 leader_election.cc:343] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.644073 8551 ts_tablet_manager.cc:1916] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.644158 8417 raft_consensus.cc:2468] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:54.644426 8083 leader_election.cc:304] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 02:59:54.644660 8562 raft_consensus.cc:2804] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:54.644730 8562 raft_consensus.cc:697] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:54.644861 8562 consensus_queue.cc:237] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.646484 8551 ts_tablet_manager.cc:1929] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.646556 8551 log.cc:1199] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ba35c8fe21f140cda8a4b1ad7568ec75
I20251028 02:59:54.646781 8551 ts_tablet_manager.cc:1950] T ba35c8fe21f140cda8a4b1ad7568ec75 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.648571 8551 tablet_replica.cc:333] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.648663 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ba35c8fe21f140cda8a4b1ad7568ec75 (table default.Table_2ad18e95c3a14e039a63bebaa46268fa [id=abb728d9aa374dce81ffa90585353fa7]) successfully deleted
I20251028 02:59:54.648721 8551 raft_consensus.cc:2243] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.648835 8551 raft_consensus.cc:2272] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.649596 8544 tablet_bootstrap.cc:492] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.649690 8544 ts_tablet_manager.cc:1403] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.000s
I20251028 02:59:54.650177 8544 raft_consensus.cc:359] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.650286 8544 raft_consensus.cc:385] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.650323 8544 raft_consensus.cc:740] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.650458 8544 consensus_queue.cc:260] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.650739 8544 ts_tablet_manager.cc:1434] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.650983 8544 tablet_bootstrap.cc:492] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.651448 8551 ts_tablet_manager.cc:1916] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.652115 8544 tablet_bootstrap.cc:654] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.654910 8550 ts_tablet_manager.cc:1929] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251028 02:59:54.655953 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3b43da9580244346ae512a7bc9b506d3 consensus state has no leader
I20251028 02:59:54.657313 8550 log.cc:1199] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/cf90b539dc8f47b0947d4e2ca859668a
I20251028 02:59:54.657788 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2de1f8c1f3564959a40963351d7d60b6" } new_table_name: "default.Table_c62970e81d90490ebe817e6ff5c68b97" modify_external_catalogs: true
I20251028 02:59:54.657966 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet e6f9b707ec0345249c3efec1c24a956e with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef353f5e715452eae1f1c191fe79d1d) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.658157 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet cdd92cd8a5194e22a91e59aaa4793135 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.658191 8550 ts_tablet_manager.cc:1950] T cf90b539dc8f47b0947d4e2ca859668a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.658963 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c7263039d1ed41bcb49853404eabc222" } modify_external_catalogs: true
I20251028 02:59:54.659085 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c7263039d1ed41bcb49853404eabc222" } modify_external_catalogs: true
I20251028 02:59:54.660040 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f56d1ad60fea4b2f9425aa0f9280df5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.660146 8545 tablet_bootstrap.cc:492] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.660235 8545 ts_tablet_manager.cc:1403] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
W20251028 02:59:54.660457 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f56d1ad60fea4b2f9425aa0f9280df5d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.661334 8545 raft_consensus.cc:359] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.661438 8545 raft_consensus.cc:385] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.661470 8545 raft_consensus.cc:740] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.661499 8544 tablet_bootstrap.cc:492] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.661576 8544 ts_tablet_manager.cc:1403] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:54.661590 8545 consensus_queue.cc:260] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.661883 8545 ts_tablet_manager.cc:1434] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:54.662070 8545 tablet_bootstrap.cc:492] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.662024 8544 raft_consensus.cc:359] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.662129 8544 raft_consensus.cc:385] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.662158 8544 raft_consensus.cc:740] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.662271 8544 consensus_queue.cc:260] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.662539 8544 ts_tablet_manager.cc:1434] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.662721 8544 tablet_bootstrap.cc:492] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.663128 8545 tablet_bootstrap.cc:654] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.663780 8544 tablet_bootstrap.cc:654] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.665082 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet cf90b539dc8f47b0947d4e2ca859668a (table default.Table_12db05bb2e3a40d594ac678a0409bf5d [id=5f82b9596e5842deb4d45e19c91629bb]) successfully deleted
I20251028 02:59:54.665331 8550 tablet_replica.cc:333] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.665448 8550 raft_consensus.cc:2243] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.665547 8550 raft_consensus.cc:2272] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.666715 8656 raft_consensus.cc:493] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.666815 8656 raft_consensus.cc:515] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.667241 8656 leader_election.cc:290] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.667462 8546 tablet_bootstrap.cc:492] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.667563 8546 ts_tablet_manager.cc:1403] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.036s user 0.000s sys 0.002s
I20251028 02:59:54.667848 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cdd92cd8a5194e22a91e59aaa4793135" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.668092 8546 raft_consensus.cc:359] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.668128 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cdd92cd8a5194e22a91e59aaa4793135" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:54.668192 8546 raft_consensus.cc:385] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.668229 8546 raft_consensus.cc:740] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.668255 8417 raft_consensus.cc:2468] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:54.668345 8546 consensus_queue.cc:260] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.668673 8546 ts_tablet_manager.cc:1434] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:54.668846 8546 tablet_bootstrap.cc:492] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.669871 8546 tablet_bootstrap.cc:654] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.670254 8551 ts_tablet_manager.cc:1929] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.670327 8551 log.cc:1199] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/d107a510c8664999901320581e885ba9
I20251028 02:59:54.670543 8551 ts_tablet_manager.cc:1950] T d107a510c8664999901320581e885ba9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.670853 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 69f1f6895a604b858b28a3e0241245de
I20251028 02:59:54.671120 8551 tablet_replica.cc:333] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.671255 8551 raft_consensus.cc:2243] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.671398 8551 raft_consensus.cc:2272] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.672024 8544 tablet_bootstrap.cc:492] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.672137 8544 ts_tablet_manager.cc:1403] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20251028 02:59:54.672398 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0e8ca626159c41aeb8e360f6e038f7a3" } new_table_name: "default.Table_476175b520b24e51a03363f567a515e8" modify_external_catalogs: true
I20251028 02:59:54.672642 8544 raft_consensus.cc:359] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.672734 8544 raft_consensus.cc:385] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.672766 8544 raft_consensus.cc:740] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.672971 8544 consensus_queue.cc:260] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.673432 8546 tablet_bootstrap.cc:492] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.673516 8546 ts_tablet_manager.cc:1403] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20251028 02:59:54.673707 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a
I20251028 02:59:54.674134 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7fa5056d9def461dab21615f8b7e8e82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.674278 8546 raft_consensus.cc:359] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.674381 8546 raft_consensus.cc:385] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.674409 8546 raft_consensus.cc:740] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
W20251028 02:59:54.674520 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7fa5056d9def461dab21615f8b7e8e82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.674530 8546 consensus_queue.cc:260] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.674952 8544 ts_tablet_manager.cc:1434] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251028 02:59:54.675151 8544 tablet_bootstrap.cc:492] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 02:59:54.675343 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a19eeac727e142579e34a301a6e2db02 consensus state has no leader
I20251028 02:59:54.675510 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.675729 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d81b3fe6073d4380882057eb97907fbb
I20251028 02:59:54.675942 7966 catalog_manager.cc:6719] ReplaceTablet: tablet d81b3fe6073d4380882057eb97907fbb deleted and replaced by tablet 923e46aca8284966980ac5b062baf829
I20251028 02:59:54.676182 8544 tablet_bootstrap.cc:654] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.676124 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_af589bb054d049a8beaed83cc51499f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.676457 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af589bb054d049a8beaed83cc51499f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.676995 8546 ts_tablet_manager.cc:1434] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251028 02:59:54.677197 8546 tablet_bootstrap.cc:492] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.677327 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_93edfbd4d0764b0aafb235f775be773f" } new_table_name: "default.Table_3d3d9ba2e032453fbbc6b2635d1b2203" modify_external_catalogs: true
I20251028 02:59:54.678716 8546 tablet_bootstrap.cc:654] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.680300 8544 tablet_bootstrap.cc:492] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.680397 8544 ts_tablet_manager.cc:1403] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20251028 02:59:54.681182 8546 tablet_bootstrap.cc:492] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.681264 8546 ts_tablet_manager.cc:1403] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20251028 02:59:54.681510 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet d107a510c8664999901320581e885ba9 (table default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]) successfully deleted
I20251028 02:59:54.681754 8546 raft_consensus.cc:359] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.681864 8546 raft_consensus.cc:385] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.681901 8546 raft_consensus.cc:740] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.682056 8546 consensus_queue.cc:260] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.682317 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 69f1f6895a604b858b28a3e0241245de with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.682456 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.682510 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet d81b3fe6073d4380882057eb97907fbb with delete_type TABLET_DATA_DELETED (replaced by tablet 923e46aca8284966980ac5b062baf829) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.682555 8544 raft_consensus.cc:359] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.682673 8544 raft_consensus.cc:385] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.682711 8544 raft_consensus.cc:740] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.682819 8544 consensus_queue.cc:260] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.683255 8083 leader_election.cc:304] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:54.683454 8551 ts_tablet_manager.cc:1916] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.683622 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 69f1f6895a604b858b28a3e0241245de with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.683653 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:54.683754 8083 leader_election.cc:343] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.683768 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet d81b3fe6073d4380882057eb97907fbb with delete_type TABLET_DATA_DELETED (replaced by tablet 923e46aca8284966980ac5b062baf829) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.683971 8562 raft_consensus.cc:2804] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:54.684019 8562 raft_consensus.cc:493] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:54.684064 8562 raft_consensus.cc:3060] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.685094 8255 tablet_service.cc:1552] Processing DeleteTablet for tablet 69f1f6895a604b858b28a3e0241245de with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.685087 8562 raft_consensus.cc:515] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.685240 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.685333 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet d81b3fe6073d4380882057eb97907fbb with delete_type TABLET_DATA_DELETED (replaced by tablet 923e46aca8284966980ac5b062baf829) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.685449 8562 leader_election.cc:290] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.685781 8562 raft_consensus.cc:493] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.685863 8562 raft_consensus.cc:515] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.686066 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cdd92cd8a5194e22a91e59aaa4793135" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:54.686182 8417 raft_consensus.cc:3060] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.686185 8562 leader_election.cc:290] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.686167 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cdd92cd8a5194e22a91e59aaa4793135" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:54.686697 8550 ts_tablet_manager.cc:1916] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.687130 8417 raft_consensus.cc:2468] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:54.687685 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9860023c7b564342ba3145eafe352beb" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:54.687804 8417 raft_consensus.cc:2468] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:54.689050 8550 ts_tablet_manager.cc:1929] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.689075 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9860023c7b564342ba3145eafe352beb" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.689122 8550 log.cc:1199] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/5fd9cc1847ff4df7a09e2e698786d515
W20251028 02:59:54.689152 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 69f1f6895a604b858b28a3e0241245de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69f1f6895a604b858b28a3e0241245de already in progress: creating tablet
I20251028 02:59:54.689189 8284 raft_consensus.cc:2468] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:54.689333 8550 ts_tablet_manager.cc:1950] T 5fd9cc1847ff4df7a09e2e698786d515 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 02:59:54.689320 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a already in progress: creating tablet
W20251028 02:59:54.689472 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet d81b3fe6073d4380882057eb97907fbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d81b3fe6073d4380882057eb97907fbb already in progress: creating tablet
I20251028 02:59:54.683254 8544 ts_tablet_manager.cc:1434] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251028 02:59:54.690435 8544 tablet_bootstrap.cc:492] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.691700 8544 tablet_bootstrap.cc:654] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.692286 8551 ts_tablet_manager.cc:1929] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.692363 8551 log.cc:1199] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/1094f4d2872449599e3eaf286def4f18
I20251028 02:59:54.692560 8551 ts_tablet_manager.cc:1950] T 1094f4d2872449599e3eaf286def4f18 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.693300 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 1094f4d2872449599e3eaf286def4f18 (table default.Table_f0712855c8944a5bbae3fc9c069b155d [id=c22cf53127924afcbc55ea4a1fd8c04f]) successfully deleted
W20251028 02:59:54.694725 8083 leader_election.cc:343] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.694845 8083 leader_election.cc:304] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:54.694976 8083 leader_election.cc:304] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 02:59:54.695277 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 5fd9cc1847ff4df7a09e2e698786d515 (table default.Table_afc8d935c16a4f0b9d3bff7efa913adf [id=55a855364d934888a7f5c31082dc6378]) successfully deleted
W20251028 02:59:54.695454 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 529b865389f94471a6849d17ff206e4c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 529b865389f94471a6849d17ff206e4c already in progress: creating tablet
W20251028 02:59:54.695545 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 69f1f6895a604b858b28a3e0241245de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69f1f6895a604b858b28a3e0241245de already in progress: creating tablet
W20251028 02:59:54.695621 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a already in progress: creating tablet
W20251028 02:59:54.695708 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet d81b3fe6073d4380882057eb97907fbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d81b3fe6073d4380882057eb97907fbb already in progress: creating tablet
I20251028 02:59:54.695829 8656 raft_consensus.cc:2804] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:54.695878 8656 raft_consensus.cc:493] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:54.695912 8656 raft_consensus.cc:3060] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.696843 8656 raft_consensus.cc:515] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:54.696947 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 529b865389f94471a6849d17ff206e4c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 529b865389f94471a6849d17ff206e4c already in progress: creating tablet
I20251028 02:59:54.697170 8551 tablet_replica.cc:333] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.697177 8656 leader_election.cc:290] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.697259 8656 raft_consensus.cc:2804] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:54.697310 8656 raft_consensus.cc:697] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:54.697319 8551 raft_consensus.cc:2243] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.697425 8551 raft_consensus.cc:2272] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.697420 8656 consensus_queue.cc:237] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.698000 8551 ts_tablet_manager.cc:1916] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.700518 8551 ts_tablet_manager.cc:1929] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.701120 7964 catalog_manager.cc:5649] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:54.701339 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (table default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]). Expected version 1 got 0
I20251028 02:59:54.702147 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 7528e7ff127c475789eda805467f0b41 with delete_type TABLET_DATA_DELETED (replaced by tablet 5428db004de74f7492c9b8999e53020e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.702351 8552 tablet_replica.cc:333] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.702517 8552 raft_consensus.cc:2243] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.702651 8552 raft_consensus.cc:2272] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.702940 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 73c20437695a4043a86faa9c98091e5e from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.703123 8546 ts_tablet_manager.cc:1434] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.022s user 0.001s sys 0.000s
I20251028 02:59:54.703172 8552 ts_tablet_manager.cc:1916] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.704159 8546 tablet_bootstrap.cc:492] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.703133 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ab414dbe9144aef8fd72a7f5a95577d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.704843 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9860023c7b564342ba3145eafe352beb" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:54.705065 8417 raft_consensus.cc:3060] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.705956 8546 tablet_bootstrap.cc:654] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.706007 8417 raft_consensus.cc:2468] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:54.706195 8551 log.cc:1199] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/e6f9b707ec0345249c3efec1c24a956e
I20251028 02:59:54.706488 8551 ts_tablet_manager.cc:1950] T e6f9b707ec0345249c3efec1c24a956e P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.706964 8083 leader_election.cc:304] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:54.707165 8562 raft_consensus.cc:2804] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:54.707232 8562 raft_consensus.cc:697] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:54.707337 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet e6f9b707ec0345249c3efec1c24a956e (table default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]) successfully deleted
I20251028 02:59:54.707376 8562 consensus_queue.cc:237] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.707656 8551 tablet_replica.cc:333] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.706800 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9860023c7b564342ba3145eafe352beb" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:54.707757 8284 raft_consensus.cc:3060] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.707901 8551 raft_consensus.cc:2243] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.708701 8284 raft_consensus.cc:2468] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:54.707047 8552 ts_tablet_manager.cc:1929] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.708911 8552 log.cc:1199] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/7528e7ff127c475789eda805467f0b41
I20251028 02:59:54.708935 8551 raft_consensus.cc:2272] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.709147 8552 ts_tablet_manager.cc:1950] T 7528e7ff127c475789eda805467f0b41 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.709604 8551 ts_tablet_manager.cc:1916] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.710196 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 7528e7ff127c475789eda805467f0b41 (table default.Table_663247205d844151bccc25218dcf39a5 [id=d40c0ee824ba423395e475f68276b5d7]) successfully deleted
I20251028 02:59:54.713153 8545 tablet_bootstrap.cc:492] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.713254 8545 ts_tablet_manager.cc:1403] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.000s
I20251028 02:59:54.713781 8545 raft_consensus.cc:359] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.713908 8545 raft_consensus.cc:385] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.713951 8545 raft_consensus.cc:740] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.714102 8545 consensus_queue.cc:260] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.714913 8551 ts_tablet_manager.cc:1929] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.716023 8545 ts_tablet_manager.cc:1434] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:54.718119 8545 tablet_bootstrap.cc:492] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.721004 8546 tablet_bootstrap.cc:492] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.721117 8546 ts_tablet_manager.cc:1403] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.001s
I20251028 02:59:54.722026 8546 raft_consensus.cc:359] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.722136 8545 tablet_bootstrap.cc:654] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.722292 8546 raft_consensus.cc:385] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.722388 8546 raft_consensus.cc:740] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.722612 8546 consensus_queue.cc:260] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.723975 8546 ts_tablet_manager.cc:1434] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:54.724233 8546 tablet_bootstrap.cc:492] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.725535 8546 tablet_bootstrap.cc:654] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.726234 8544 tablet_bootstrap.cc:492] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.726401 8544 ts_tablet_manager.cc:1403] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.002s
W20251028 02:59:54.726913 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f08a6e30211a4d59934c36c7a26ac983 consensus state has no leader
I20251028 02:59:54.726976 8544 raft_consensus.cc:359] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.727092 8544 raft_consensus.cc:385] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.727125 8544 raft_consensus.cc:740] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.727241 8544 consensus_queue.cc:260] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.727537 8544 ts_tablet_manager.cc:1434] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.727730 8544 tablet_bootstrap.cc:492] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.728901 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c048520fe7b248e586e44a2047ffb239
I20251028 02:59:54.729642 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet c048520fe7b248e586e44a2047ffb239 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.729861 8552 tablet_replica.cc:333] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.730016 8552 raft_consensus.cc:2243] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:54.730237 8552 raft_consensus.cc:2272] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.730551 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e4f1369ad6ad4110b5854544d4a266bd" } modify_external_catalogs: true
I20251028 02:59:54.730675 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e4f1369ad6ad4110b5854544d4a266bd" } modify_external_catalogs: true
I20251028 02:59:54.730803 8552 ts_tablet_manager.cc:1916] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.730809 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6aba0ca2a3074aac8a0bf7f2da936e1a" } modify_external_catalogs: true
I20251028 02:59:54.730901 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6aba0ca2a3074aac8a0bf7f2da936e1a" } modify_external_catalogs: true
I20251028 02:59:54.731529 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet d81b3fe6073d4380882057eb97907fbb from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.731880 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_9f6c006d57c74ddd9b5297aa3d310af4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.732276 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f6c006d57c74ddd9b5297aa3d310af4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.732996 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet c048520fe7b248e586e44a2047ffb239 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.733150 8544 tablet_bootstrap.cc:654] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.733304 8552 ts_tablet_manager.cc:1929] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:54.733366 8552 log.cc:1199] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c048520fe7b248e586e44a2047ffb239
I20251028 02:59:54.733647 8552 ts_tablet_manager.cc:1950] T c048520fe7b248e586e44a2047ffb239 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 02:59:54.733937 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aaaec176a3884a79a55bb74397a7dc9c has no consensus state
I20251028 02:59:54.734170 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_8924bd5725814f70bc3834b45c618074"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.734290 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 7801d57550dc40c39f2d3cf7e11e3713 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:54.734535 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8924bd5725814f70bc3834b45c618074 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.735142 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4d8bf6511df640c582ce134a8c146266"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.735507 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d8bf6511df640c582ce134a8c146266 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.736766 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]) successfully deleted
I20251028 02:59:54.737218 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet c083ca984a7347a6b5bb0f3a949a5f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.738404 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet ba35c8fe21f140cda8a4b1ad7568ec75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.738559 8255 tablet_service.cc:1552] Processing DeleteTablet for tablet a3096fde2ac94ab197ce5e659c4d2be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.738699 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet d107a510c8664999901320581e885ba9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a2ebe28dbd2439e963319ec6e515635) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.738854 8552 tablet_replica.cc:333] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.739003 8552 raft_consensus.cc:2243] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.739128 8552 raft_consensus.cc:2272] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.739315 7966 catalog_manager.cc:5649] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:54.739584 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 56c8969da9dc464d9b463205e3a27965 (table default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]). Expected version 1 got 0
I20251028 02:59:54.739694 8552 ts_tablet_manager.cc:1916] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.739827 7966 catalog_manager.cc:5649] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:54.741623 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 73c20437695a4043a86faa9c98091e5e
I20251028 02:59:54.741894 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 73c20437695a4043a86faa9c98091e5e deleted and replaced by tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a
I20251028 02:59:54.742439 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 73c20437695a4043a86faa9c98091e5e with delete_type TABLET_DATA_DELETED (replaced by tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.742600 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 73c20437695a4043a86faa9c98091e5e with delete_type TABLET_DATA_DELETED (replaced by tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.743484 8552 ts_tablet_manager.cc:1929] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.743549 8552 log.cc:1199] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c083ca984a7347a6b5bb0f3a949a5f75
I20251028 02:59:54.743552 7969 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_0e8ca626159c41aeb8e360f6e038f7a3 [id=eb0e103661fc4bc7b27bb78fc333fd56]). Expected version 1 got 0
I20251028 02:59:54.743789 8552 ts_tablet_manager.cc:1950] T c083ca984a7347a6b5bb0f3a949a5f75 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.743959 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet c048520fe7b248e586e44a2047ffb239 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.743960 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet a3096fde2ac94ab197ce5e659c4d2be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.744148 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 7528e7ff127c475789eda805467f0b41 with delete_type TABLET_DATA_DELETED (replaced by tablet 5428db004de74f7492c9b8999e53020e) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.744190 8550 tablet_replica.cc:333] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.744243 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet d107a510c8664999901320581e885ba9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2a2ebe28dbd2439e963319ec6e515635) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.744158 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 73c20437695a4043a86faa9c98091e5e with delete_type TABLET_DATA_DELETED (replaced by tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.744351 8550 raft_consensus.cc:2243] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.744491 8550 raft_consensus.cc:2272] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.745196 8550 ts_tablet_manager.cc:1916] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.745285 8552 tablet_replica.cc:333] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.745405 8552 raft_consensus.cc:2243] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.745504 8552 raft_consensus.cc:2272] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.746287 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c083ca984a7347a6b5bb0f3a949a5f75 (table default.Table_d1b772f7ad114598bf453e2265c5e436 [id=73579e6e9da94fa699b3cb5bb5cec89e]) successfully deleted
I20251028 02:59:54.747638 8550 ts_tablet_manager.cc:1929] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:54.747709 8550 log.cc:1199] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c048520fe7b248e586e44a2047ffb239
I20251028 02:59:54.747957 8550 ts_tablet_manager.cc:1950] T c048520fe7b248e586e44a2047ffb239 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.748616 8550 tablet_replica.cc:333] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.748751 8550 raft_consensus.cc:2243] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.748847 8550 raft_consensus.cc:2272] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.749501 8545 tablet_bootstrap.cc:492] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.749595 8545 ts_tablet_manager.cc:1403] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20251028 02:59:54.750191 8545 raft_consensus.cc:359] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.750303 8545 raft_consensus.cc:385] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.750337 8545 raft_consensus.cc:740] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.750474 8545 consensus_queue.cc:260] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.752557 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]) successfully deleted
I20251028 02:59:54.752846 9048 tablet.cc:1722] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:54.753433 9122 tablet.cc:1722] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:54.752187 8545 ts_tablet_manager.cc:1434] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:54.755764 8545 tablet_bootstrap.cc:492] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.756106 8552 ts_tablet_manager.cc:1916] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.756800 8545 tablet_bootstrap.cc:654] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.757481 8417 raft_consensus.cc:1275] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:54.757527 8551 log.cc:1199] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/cdd92cd8a5194e22a91e59aaa4793135
I20251028 02:59:54.757750 8551 ts_tablet_manager.cc:1950] T cdd92cd8a5194e22a91e59aaa4793135 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.758028 8145 raft_consensus.cc:1275] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:54.758424 8551 tablet_replica.cc:333] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.758399 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet cdd92cd8a5194e22a91e59aaa4793135 (table default.Table_54bb44e4b3aa4babb80308231391809c [id=2bd44c4bfd2741c7bec6f973ffc7682d]) successfully deleted
I20251028 02:59:54.758570 8551 raft_consensus.cc:2243] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.758554 8553 consensus_queue.cc:1048] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.758679 8551 raft_consensus.cc:2272] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.759346 8553 consensus_queue.cc:1048] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.760056 9023 tablet.cc:1722] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:54.764986 8550 ts_tablet_manager.cc:1916] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.767902 8551 ts_tablet_manager.cc:1916] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.770332 8551 ts_tablet_manager.cc:1929] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.770426 8551 log.cc:1199] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/69f1f6895a604b858b28a3e0241245de
I20251028 02:59:54.772848 8552 ts_tablet_manager.cc:1929] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.772936 8552 log.cc:1199] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/ba35c8fe21f140cda8a4b1ad7568ec75
I20251028 02:59:54.773149 8552 ts_tablet_manager.cc:1950] T ba35c8fe21f140cda8a4b1ad7568ec75 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.773645 8551 ts_tablet_manager.cc:1950] T 69f1f6895a604b858b28a3e0241245de P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.774371 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 69f1f6895a604b858b28a3e0241245de (table default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]) successfully deleted
I20251028 02:59:54.779122 9328 log.cc:826] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Log is configured to *not* fsync() on all Append() calls
I20251028 02:59:54.779462 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet ba35c8fe21f140cda8a4b1ad7568ec75 (table default.Table_2ad18e95c3a14e039a63bebaa46268fa [id=abb728d9aa374dce81ffa90585353fa7]) successfully deleted
I20251028 02:59:54.779579 8552 tablet_replica.cc:333] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.779778 8552 raft_consensus.cc:2243] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.779973 8552 raft_consensus.cc:2272] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:54.780030 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a already in progress: creating tablet
W20251028 02:59:54.780611 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet d81b3fe6073d4380882057eb97907fbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d81b3fe6073d4380882057eb97907fbb already in progress: creating tablet
I20251028 02:59:54.780802 8551 tablet_replica.cc:333] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.780977 8550 ts_tablet_manager.cc:1929] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.780988 8551 raft_consensus.cc:2243] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.781096 8550 log.cc:1199] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a3096fde2ac94ab197ce5e659c4d2be0
I20251028 02:59:54.781164 8551 raft_consensus.cc:2272] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.781327 8550 ts_tablet_manager.cc:1950] T a3096fde2ac94ab197ce5e659c4d2be0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.781790 8551 ts_tablet_manager.cc:1916] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.781984 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a3096fde2ac94ab197ce5e659c4d2be0 (table default.Table_6d8f283a536e46fcbf3d8abcf9d703bb [id=a6d84958fd654087940c9806e2e3fbc0]) successfully deleted
I20251028 02:59:54.782207 8550 tablet_replica.cc:333] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.784104 8147 raft_consensus.cc:3060] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.784816 8551 ts_tablet_manager.cc:1929] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.785337 8552 ts_tablet_manager.cc:1916] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.784121 8550 raft_consensus.cc:2243] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.785871 8553 consensus_queue.cc:1048] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.785997 8550 raft_consensus.cc:2272] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.787791 8552 ts_tablet_manager.cc:1929] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.787946 8551 log.cc:1199] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2ab414dbe9144aef8fd72a7f5a95577d
I20251028 02:59:54.788256 8551 ts_tablet_manager.cc:1950] T 2ab414dbe9144aef8fd72a7f5a95577d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.788638 8550 ts_tablet_manager.cc:1916] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.789342 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2ab414dbe9144aef8fd72a7f5a95577d (table default.Table_44487777ef26464c9cbbb195199247a0 [id=f04d44307e3c4c07a2403477d11e9fcf]) successfully deleted
I20251028 02:59:54.789655 8551 tablet_replica.cc:333] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.789891 8551 raft_consensus.cc:2243] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.790093 8551 raft_consensus.cc:2272] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.790731 8551 ts_tablet_manager.cc:1916] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.792964 8551 ts_tablet_manager.cc:1929] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:54.793123 8552 log.cc:1199] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/a3096fde2ac94ab197ce5e659c4d2be0
I20251028 02:59:54.793426 8552 ts_tablet_manager.cc:1950] T a3096fde2ac94ab197ce5e659c4d2be0 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.794205 8552 tablet_replica.cc:333] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.794390 8552 raft_consensus.cc:2243] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.794540 8552 raft_consensus.cc:2272] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.794705 8551 log.cc:1199] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c048520fe7b248e586e44a2047ffb239
I20251028 02:59:54.794982 8551 ts_tablet_manager.cc:1950] T c048520fe7b248e586e44a2047ffb239 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.795516 8656 raft_consensus.cc:493] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.795621 8656 raft_consensus.cc:515] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.795686 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet c048520fe7b248e586e44a2047ffb239 (table default.Table_c7263039d1ed41bcb49853404eabc222 [id=f2d1a37b2cac4447b19d61632e988672]) successfully deleted
I20251028 02:59:54.795972 8551 tablet_replica.cc:333] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.796072 8656 leader_election.cc:290] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.796110 8551 raft_consensus.cc:2243] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.796211 8551 raft_consensus.cc:2272] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.796660 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "69f1f6895a604b858b28a3e0241245de" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.796710 8551 ts_tablet_manager.cc:1916] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.796772 8552 ts_tablet_manager.cc:1916] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:54.797263 8083 leader_election.cc:343] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:54.797394 8083 leader_election.cc:343] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 69f1f6895a604b858b28a3e0241245de
I20251028 02:59:54.797437 8083 leader_election.cc:304] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:54.797609 8656 raft_consensus.cc:2749] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.797583 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet a3096fde2ac94ab197ce5e659c4d2be0 (table default.Table_6d8f283a536e46fcbf3d8abcf9d703bb [id=a6d84958fd654087940c9806e2e3fbc0]) successfully deleted
I20251028 02:59:54.797962 8656 raft_consensus.cc:493] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.798045 8656 raft_consensus.cc:515] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.798429 8656 leader_election.cc:290] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.799158 9122 tablet.cc:1722] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251028 02:59:54.799654 8083 leader_election.cc:343] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: c083ca984a7347a6b5bb0f3a949a5f75
W20251028 02:59:54.799755 8083 leader_election.cc:343] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: c083ca984a7347a6b5bb0f3a949a5f75
I20251028 02:59:54.799805 8083 leader_election.cc:304] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:54.800654 9023 tablet.cc:1722] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.801728 8656 raft_consensus.cc:2749] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.802964 8551 ts_tablet_manager.cc:1929] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.803040 8551 log.cc:1199] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/7801d57550dc40c39f2d3cf7e11e3713
I20251028 02:59:54.803282 8551 ts_tablet_manager.cc:1950] T 7801d57550dc40c39f2d3cf7e11e3713 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.805320 8545 tablet_bootstrap.cc:492] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.805414 8545 ts_tablet_manager.cc:1403] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.050s user 0.001s sys 0.000s
I20251028 02:59:54.806025 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 56c8969da9dc464d9b463205e3a27965 (table default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]). Expected version 1 got 0
I20251028 02:59:54.805969 8545 raft_consensus.cc:359] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.806102 8545 raft_consensus.cc:385] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.806133 8545 raft_consensus.cc:740] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.806268 8545 consensus_queue.cc:260] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.806548 8545 ts_tablet_manager.cc:1434] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.806754 8545 tablet_bootstrap.cc:492] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.808722 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet c083ca984a7347a6b5bb0f3a949a5f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:52 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.808972 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet a3096fde2ac94ab197ce5e659c4d2be0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.809101 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet e6f9b707ec0345249c3efec1c24a956e with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef353f5e715452eae1f1c191fe79d1d) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.809875 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 69f1f6895a604b858b28a3e0241245de with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.810004 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 1094f4d2872449599e3eaf286def4f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.810097 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ab414dbe9144aef8fd72a7f5a95577d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.810359 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet cdd92cd8a5194e22a91e59aaa4793135 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.810487 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet ba35c8fe21f140cda8a4b1ad7568ec75 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.812443 8562 raft_consensus.cc:493] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.812541 8562 raft_consensus.cc:515] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.812902 8562 leader_election.cc:290] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.813205 8545 tablet_bootstrap.cc:654] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.813441 8469 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:54.814404 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 7801d57550dc40c39f2d3cf7e11e3713 (table default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]) successfully deleted
W20251028 02:59:54.815006 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 73c20437695a4043a86faa9c98091e5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73c20437695a4043a86faa9c98091e5e already in progress: creating tablet
W20251028 02:59:54.815451 8083 leader_election.cc:343] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: ba35c8fe21f140cda8a4b1ad7568ec75
W20251028 02:59:54.815578 8083 leader_election.cc:343] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: ba35c8fe21f140cda8a4b1ad7568ec75
I20251028 02:59:54.815618 8083 leader_election.cc:304] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:54.815765 8562 raft_consensus.cc:2749] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.817322 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9f1c1e14665c4712bffa439ab98b2a6e
I20251028 02:59:54.817595 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d59acfb0e12643ebbb406dbda7bef29b
I20251028 02:59:54.817967 8196 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:54.818303 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 9f1c1e14665c4712bffa439ab98b2a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.818661 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet d59acfb0e12643ebbb406dbda7bef29b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.818828 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 9f1c1e14665c4712bffa439ab98b2a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.818970 8255 tablet_service.cc:1552] Processing DeleteTablet for tablet d59acfb0e12643ebbb406dbda7bef29b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.819433 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet d59acfb0e12643ebbb406dbda7bef29b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.819433 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 9f1c1e14665c4712bffa439ab98b2a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.819641 8469 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
W20251028 02:59:54.819950 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet d59acfb0e12643ebbb406dbda7bef29b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d59acfb0e12643ebbb406dbda7bef29b already in progress: creating tablet
I20251028 02:59:54.820153 8551 tablet_replica.cc:333] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.820291 8551 raft_consensus.cc:2243] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.820421 8551 raft_consensus.cc:2272] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.820912 8551 ts_tablet_manager.cc:1916] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.822068 8196 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:54.823412 9048 tablet.cc:1722] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.824862 8551 ts_tablet_manager.cc:1929] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:54.825012 8551 log.cc:1199] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9f1c1e14665c4712bffa439ab98b2a6e
I20251028 02:59:54.826279 8544 tablet_bootstrap.cc:492] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.826377 8544 ts_tablet_manager.cc:1403] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.099s user 0.003s sys 0.000s
I20251028 02:59:54.826872 8544 raft_consensus.cc:359] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.826996 8544 raft_consensus.cc:385] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.827041 8544 raft_consensus.cc:740] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.827169 8551 ts_tablet_manager.cc:1950] T 9f1c1e14665c4712bffa439ab98b2a6e P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.827160 8544 consensus_queue.cc:260] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.827498 8544 ts_tablet_manager.cc:1434] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.827690 8544 tablet_bootstrap.cc:492] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.828781 8544 tablet_bootstrap.cc:654] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.829900 8552 ts_tablet_manager.cc:1929] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.829967 8552 log.cc:1199] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/d107a510c8664999901320581e885ba9
I20251028 02:59:54.830181 8552 ts_tablet_manager.cc:1950] T d107a510c8664999901320581e885ba9 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.830849 8552 tablet_replica.cc:333] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.830981 8552 raft_consensus.cc:2243] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:54.831103 8333 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:54.831175 8552 raft_consensus.cc:2272] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.831563 8552 ts_tablet_manager.cc:1916] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.832563 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet d107a510c8664999901320581e885ba9 (table default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]) successfully deleted
W20251028 02:59:54.832751 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 73c20437695a4043a86faa9c98091e5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73c20437695a4043a86faa9c98091e5e already in progress: creating tablet
I20251028 02:59:54.832861 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 99c37c774ae2458989377a8bac94f4e8 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.833016 8553 consensus_queue.cc:1048] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.837090 8550 ts_tablet_manager.cc:1929] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.837181 8550 log.cc:1199] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/7528e7ff127c475789eda805467f0b41
I20251028 02:59:54.837236 8552 ts_tablet_manager.cc:1929] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:54.837311 8552 log.cc:1199] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9f1c1e14665c4712bffa439ab98b2a6e
I20251028 02:59:54.837388 8550 ts_tablet_manager.cc:1950] T 7528e7ff127c475789eda805467f0b41 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.837474 8545 tablet_bootstrap.cc:492] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.837570 8545 ts_tablet_manager.cc:1403] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20251028 02:59:54.838121 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 7528e7ff127c475789eda805467f0b41 (table default.Table_663247205d844151bccc25218dcf39a5 [id=d40c0ee824ba423395e475f68276b5d7]) successfully deleted
I20251028 02:59:54.838106 8545 raft_consensus.cc:359] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.838215 8545 raft_consensus.cc:385] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.838245 8545 raft_consensus.cc:740] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.838375 8545 consensus_queue.cc:260] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.837560 8552 ts_tablet_manager.cc:1950] T 9f1c1e14665c4712bffa439ab98b2a6e P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.838744 8545 ts_tablet_manager.cc:1434] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.838943 8545 tablet_bootstrap.cc:492] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.840646 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9f1c1e14665c4712bffa439ab98b2a6e (table default.Table_6aba0ca2a3074aac8a0bf7f2da936e1a [id=3eb0b70129be40e598874d492735ba51]) successfully deleted
W20251028 02:59:54.840868 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet d59acfb0e12643ebbb406dbda7bef29b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d59acfb0e12643ebbb406dbda7bef29b already in progress: creating tablet
I20251028 02:59:54.841832 8545 tablet_bootstrap.cc:654] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.844077 8965 consensus_queue.cc:1048] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.848217 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]). Expected version 1 got 0
I20251028 02:59:54.848984 8562 raft_consensus.cc:493] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.849095 8562 raft_consensus.cc:515] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.849462 8562 leader_election.cc:290] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.850654 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 900973d832c24b62a4554ae9a219cfd8 (DEFAULT_TABLE table=default.Table_dca026923a844b5faf18d96820d52dbd [id=c4aadbebe43f451ea22467feaa6dafbb]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:54.850731 8083 leader_election.cc:343] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: d107a510c8664999901320581e885ba9
I20251028 02:59:54.850987 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 900973d832c24b62a4554ae9a219cfd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.851846 8550 tablet_replica.cc:333] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.852702 8550 raft_consensus.cc:2243] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.852828 8550 raft_consensus.cc:2272] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.853482 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 83059074c365440b84e39cf7a5553eb2 (DEFAULT_TABLE table=default.Table_2b7f5657d1534447a3a29a01698581e1 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.853682 8550 ts_tablet_manager.cc:1916] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.853729 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83059074c365440b84e39cf7a5553eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.854245 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 99c37c774ae2458989377a8bac94f4e8
I20251028 02:59:54.854529 7961 catalog_manager.cc:6719] ReplaceTablet: tablet 99c37c774ae2458989377a8bac94f4e8 deleted and replaced by tablet 3f04c867f0ce4ab99f980cca063abc30
I20251028 02:59:54.854526 8553 consensus_queue.cc:1048] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:54.855809 8083 leader_election.cc:343] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: d107a510c8664999901320581e885ba9
I20251028 02:59:54.855885 8083 leader_election.cc:304] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:54.855978 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 73a45776f8964e68b79bc973097223a3 (DEFAULT_TABLE table=default.Table_bd1f7ca82cb84916aa7e1cbc5d217089 [id=a9b54811e6ba4461a372d76424f07b4f]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:54.855959 8083 raft_consensus.cc:2686] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251028 02:59:54.856218 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a45776f8964e68b79bc973097223a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.857544 8546 tablet_bootstrap.cc:492] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.857631 8546 ts_tablet_manager.cc:1403] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.133s user 0.001s sys 0.001s
I20251028 02:59:54.857924 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9f1c1e14665c4712bffa439ab98b2a6e (table default.Table_6aba0ca2a3074aac8a0bf7f2da936e1a [id=3eb0b70129be40e598874d492735ba51]) successfully deleted
I20251028 02:59:54.858333 8546 raft_consensus.cc:359] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.858376 8125 tablet_service.cc:1505] Processing CreateTablet for tablet b4c612bc752847c2be0547cedfd665ae (DEFAULT_TABLE table=default.Table_efbcb24780904b469cfe628420a6addd [id=a82bb1432b8546188bb85f1405f123ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.858449 8546 raft_consensus.cc:385] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.858482 8546 raft_consensus.cc:740] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.858603 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4c612bc752847c2be0547cedfd665ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.858594 8546 consensus_queue.cc:260] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.858894 8546 ts_tablet_manager.cc:1434] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251028 02:59:54.859084 8546 tablet_bootstrap.cc:492] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.860749 8546 tablet_bootstrap.cc:654] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.860728 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 1de3acf65f744368ab31dc1c82ac554f (DEFAULT_TABLE table=default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.861618 8253 tablet_service.cc:1505] Processing CreateTablet for tablet b4c612bc752847c2be0547cedfd665ae (DEFAULT_TABLE table=default.Table_efbcb24780904b469cfe628420a6addd [id=a82bb1432b8546188bb85f1405f123ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.861886 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4c612bc752847c2be0547cedfd665ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.861846 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 900973d832c24b62a4554ae9a219cfd8 (DEFAULT_TABLE table=default.Table_dca026923a844b5faf18d96820d52dbd [id=c4aadbebe43f451ea22467feaa6dafbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.862063 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 1de3acf65f744368ab31dc1c82ac554f (DEFAULT_TABLE table=default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.862340 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 99c37c774ae2458989377a8bac94f4e8 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f04c867f0ce4ab99f980cca063abc30) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.862124 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 900973d832c24b62a4554ae9a219cfd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.862484 8401 tablet_service.cc:1505] Processing CreateTablet for tablet b4c612bc752847c2be0547cedfd665ae (DEFAULT_TABLE table=default.Table_efbcb24780904b469cfe628420a6addd [id=a82bb1432b8546188bb85f1405f123ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.862696 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4c612bc752847c2be0547cedfd665ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.863020 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 900973d832c24b62a4554ae9a219cfd8 (DEFAULT_TABLE table=default.Table_dca026923a844b5faf18d96820d52dbd [id=c4aadbebe43f451ea22467feaa6dafbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.863229 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 900973d832c24b62a4554ae9a219cfd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.863389 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1de3acf65f744368ab31dc1c82ac554f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.864159 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (DEFAULT_TABLE table=default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.864379 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240b7da6f46c4c5ab8087ab1dd6275c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.862322 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 83059074c365440b84e39cf7a5553eb2 (DEFAULT_TABLE table=default.Table_2b7f5657d1534447a3a29a01698581e1 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.864979 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83059074c365440b84e39cf7a5553eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.865362 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 83059074c365440b84e39cf7a5553eb2 (DEFAULT_TABLE table=default.Table_2b7f5657d1534447a3a29a01698581e1 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.865564 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83059074c365440b84e39cf7a5553eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.866382 8393 tablet_service.cc:1505] Processing CreateTablet for tablet 84c3602bee3b4334a2a30daec986791d (DEFAULT_TABLE table=default.Table_5a5158617c6246c29742d950e6c2f0cd [id=6e93c9556d8a4644a0dff15ba3e72afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.866596 8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84c3602bee3b4334a2a30daec986791d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.867643 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 73a45776f8964e68b79bc973097223a3 (DEFAULT_TABLE table=default.Table_bd1f7ca82cb84916aa7e1cbc5d217089 [id=a9b54811e6ba4461a372d76424f07b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.867871 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a45776f8964e68b79bc973097223a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.869686 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b
I20251028 02:59:54.869902 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 1de3acf65f744368ab31dc1c82ac554f (DEFAULT_TABLE table=default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.869992 7963 catalog_manager.cc:6719] ReplaceTablet: tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b deleted and replaced by tablet 55e72a7f8aa842049da7aeaaa7001a28
I20251028 02:59:54.870118 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1de3acf65f744368ab31dc1c82ac554f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.871685 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b with delete_type TABLET_DATA_DELETED (replaced by tablet 55e72a7f8aa842049da7aeaaa7001a28) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.872234 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 99c37c774ae2458989377a8bac94f4e8 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f04c867f0ce4ab99f980cca063abc30) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.872501 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (DEFAULT_TABLE table=default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.872704 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240b7da6f46c4c5ab8087ab1dd6275c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.874754 7960 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_f9254be0618c49f8a6a36fb9fc10354a [id=86c2f59df3dd4d2195d18835dceff79b] alter complete (version 1)
I20251028 02:59:54.875072 8552 tablet_replica.cc:333] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.875108 8256 tablet_service.cc:1505] Processing CreateTablet for tablet aaaec176a3884a79a55bb74397a7dc9c (DEFAULT_TABLE table=default.Table_3d3d9ba2e032453fbbc6b2635d1b2203 [id=51fb1b3116c14113b2e14ced3ac17262]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.875221 8552 raft_consensus.cc:2243] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.875324 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaaec176a3884a79a55bb74397a7dc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.877592 8552 raft_consensus.cc:2272] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.877677 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 84c3602bee3b4334a2a30daec986791d (DEFAULT_TABLE table=default.Table_5a5158617c6246c29742d950e6c2f0cd [id=6e93c9556d8a4644a0dff15ba3e72afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.877897 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84c3602bee3b4334a2a30daec986791d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.877995 8552 ts_tablet_manager.cc:1916] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.878304 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 73a45776f8964e68b79bc973097223a3 (DEFAULT_TABLE table=default.Table_bd1f7ca82cb84916aa7e1cbc5d217089 [id=a9b54811e6ba4461a372d76424f07b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.878458 7969 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_476175b520b24e51a03363f567a515e8 [id=eb0e103661fc4bc7b27bb78fc333fd56] alter complete (version 2)
I20251028 02:59:54.878564 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a45776f8964e68b79bc973097223a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.881064 8546 tablet_bootstrap.cc:492] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.881072 8396 tablet_service.cc:1505] Processing CreateTablet for tablet aaaec176a3884a79a55bb74397a7dc9c (DEFAULT_TABLE table=default.Table_3d3d9ba2e032453fbbc6b2635d1b2203 [id=51fb1b3116c14113b2e14ced3ac17262]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.881163 8546 ts_tablet_manager.cc:1403] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20251028 02:59:54.881330 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaaec176a3884a79a55bb74397a7dc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.881691 8546 raft_consensus.cc:359] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.881793 8546 raft_consensus.cc:385] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.881820 8546 raft_consensus.cc:740] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.881938 8546 consensus_queue.cc:260] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.882190 8546 ts_tablet_manager.cc:1434] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:54.882386 8546 tablet_bootstrap.cc:492] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.882678 8550 ts_tablet_manager.cc:1929] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.882748 8550 log.cc:1199] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/d107a510c8664999901320581e885ba9
I20251028 02:59:54.882985 8550 ts_tablet_manager.cc:1950] T d107a510c8664999901320581e885ba9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.884886 8546 tablet_bootstrap.cc:654] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.855952 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 99c37c774ae2458989377a8bac94f4e8 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f04c867f0ce4ab99f980cca063abc30) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.885746 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1de3acf65f744368ab31dc1c82ac554f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.885824 8544 tablet_bootstrap.cc:492] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.885906 8544 ts_tablet_manager.cc:1403] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.058s user 0.002s sys 0.000s
I20251028 02:59:54.885928 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]). Expected version 1 got 0
I20251028 02:59:54.886345 8544 raft_consensus.cc:359] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.886440 8544 raft_consensus.cc:385] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.886466 8544 raft_consensus.cc:740] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.886574 8544 consensus_queue.cc:260] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.886819 8544 ts_tablet_manager.cc:1434] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.892180 8546 tablet_bootstrap.cc:492] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.892283 8546 ts_tablet_manager.cc:1403] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:54.892731 8546 raft_consensus.cc:359] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.892832 8546 raft_consensus.cc:385] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.892861 8546 raft_consensus.cc:740] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.892972 8546 consensus_queue.cc:260] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.893227 8546 ts_tablet_manager.cc:1434] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.893795 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]). Expected version 1 got 0
I20251028 02:59:54.894631 8546 tablet_bootstrap.cc:492] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.895510 8417 raft_consensus.cc:1275] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:54.895675 8546 tablet_bootstrap.cc:654] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.895869 8284 raft_consensus.cc:1275] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:54.896173 8562 consensus_queue.cc:1048] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.896448 8562 consensus_queue.cc:1048] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:54.897126 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet de38b2d54cec4f4e892047f4650f611b with delete_type TABLET_DATA_DELETED (replaced by tablet b93581f0eb7e499597c4e5d2f6c058aa) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.898711 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]). Expected version 1 got 0
I20251028 02:59:54.900041 8393 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b with delete_type TABLET_DATA_DELETED (replaced by tablet 55e72a7f8aa842049da7aeaaa7001a28) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.900110 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 69f1f6895a604b858b28a3e0241245de with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.900312 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet de38b2d54cec4f4e892047f4650f611b with delete_type TABLET_DATA_DELETED (replaced by tablet b93581f0eb7e499597c4e5d2f6c058aa) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.901854 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet d81b3fe6073d4380882057eb97907fbb with delete_type TABLET_DATA_DELETED (replaced by tablet 923e46aca8284966980ac5b062baf829) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:54.902331 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 7801d57550dc40c39f2d3cf7e11e3713 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.903533 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (table default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]). Expected version 1 got 0
I20251028 02:59:54.904491 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 4902bc3398754802a5c571036cd19fa7 (DEFAULT_TABLE table=default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.904839 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4902bc3398754802a5c571036cd19fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.904965 7969 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (table default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]). Expected version 1 got 0
I20251028 02:59:54.907856 8552 ts_tablet_manager.cc:1929] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:54.903285 8125 tablet_service.cc:1505] Processing CreateTablet for tablet aaaec176a3884a79a55bb74397a7dc9c (DEFAULT_TABLE table=default.Table_3d3d9ba2e032453fbbc6b2635d1b2203 [id=51fb1b3116c14113b2e14ced3ac17262]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.903533 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 84c3602bee3b4334a2a30daec986791d (DEFAULT_TABLE table=default.Table_5a5158617c6246c29742d950e6c2f0cd [id=6e93c9556d8a4644a0dff15ba3e72afa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.904263 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 4902bc3398754802a5c571036cd19fa7 (DEFAULT_TABLE table=default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.911609 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 4902bc3398754802a5c571036cd19fa7 (DEFAULT_TABLE table=default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.911931 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4902bc3398754802a5c571036cd19fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.912184 8552 log.cc:1199] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/99c37c774ae2458989377a8bac94f4e8
I20251028 02:59:54.912461 8552 ts_tablet_manager.cc:1950] T 99c37c774ae2458989377a8bac94f4e8 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.915177 8654 mvcc.cc:204] Tried to move back new op lower bound from 7215597137481527296 to 7215597136187920384. Current Snapshot: MvccSnapshot[applied={T|T < 7215597137481527296}]
I20251028 02:59:54.915349 9023 tablet.cc:1722] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.915437 9048 tablet.cc:1722] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.910239 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b with delete_type TABLET_DATA_DELETED (replaced by tablet 55e72a7f8aa842049da7aeaaa7001a28) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.915829 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (DEFAULT_TABLE table=default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.916968 8546 tablet_bootstrap.cc:492] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.917057 8546 ts_tablet_manager.cc:1403] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20251028 02:59:54.918000 8546 raft_consensus.cc:359] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.918112 8546 raft_consensus.cc:385] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.918145 8546 raft_consensus.cc:740] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.918263 8546 consensus_queue.cc:260] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.918576 8546 ts_tablet_manager.cc:1434] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:54.918785 8546 tablet_bootstrap.cc:492] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.919452 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 99c37c774ae2458989377a8bac94f4e8 (table default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]) successfully deleted
I20251028 02:59:54.919921 8546 tablet_bootstrap.cc:654] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.920019 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f3795491c9e547af98943428398101b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.920420 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3795491c9e547af98943428398101b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.920571 8556 raft_consensus.cc:493] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.920640 8556 raft_consensus.cc:515] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.920805 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_48dcb7bf58c748a28999785ae72f248a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.920997 8556 leader_election.cc:290] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:54.921156 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48dcb7bf58c748a28999785ae72f248a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.921205 8551 tablet_replica.cc:333] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.921236 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_640f9dd799c74ce4bbbc0839fd52b04f" } modify_external_catalogs: true
I20251028 02:59:54.921324 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_640f9dd799c74ce4bbbc0839fd52b04f" } modify_external_catalogs: true
I20251028 02:59:54.921465 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1306f4d4723b45fbb688bfccebd665a5" } new_table_name: "default.Table_8f6fdbd222f4456596107350e6c3c6e1" modify_external_catalogs: true
I20251028 02:59:54.921475 8551 raft_consensus.cc:2243] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
W20251028 02:59:54.921692 8220 leader_election.cc:343] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: cdd92cd8a5194e22a91e59aaa4793135
I20251028 02:59:54.921859 8551 raft_consensus.cc:2272] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.921878 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a90dbf2ca45f4e3d8e9352c7eb8b3d99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:54.922194 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a90dbf2ca45f4e3d8e9352c7eb8b3d99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.922269 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e3ed44682141426f856f581db3e481af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.922704 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cdd92cd8a5194e22a91e59aaa4793135" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:54.922785 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_13cf93a2eb1f41379907b8639554e44a" } modify_external_catalogs: true
I20251028 02:59:54.922861 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_13cf93a2eb1f41379907b8639554e44a" } modify_external_catalogs: true
I20251028 02:59:54.922653 8551 ts_tablet_manager.cc:1916] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.923190 8218 leader_election.cc:304] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.923223 8544 tablet_bootstrap.cc:492] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.923446 8965 raft_consensus.cc:2749] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251028 02:59:54.923854 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e3ed44682141426f856f581db3e481af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.924176 8550 tablet_replica.cc:333] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.924310 8544 tablet_bootstrap.cc:654] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.924330 8550 raft_consensus.cc:2243] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.924427 8550 raft_consensus.cc:2272] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.924832 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet d107a510c8664999901320581e885ba9 (table default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]) successfully deleted
I20251028 02:59:54.921169 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_12db05bb2e3a40d594ac678a0409bf5d" } new_table_name: "default.Table_fd3daefb8df048e7bea5949c30da1e1f" modify_external_catalogs: true
W20251028 02:59:54.924995 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 73c20437695a4043a86faa9c98091e5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73c20437695a4043a86faa9c98091e5e already in progress: creating tablet
I20251028 02:59:54.925397 8550 ts_tablet_manager.cc:1916] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.926519 8545 tablet_bootstrap.cc:492] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.926618 8545 ts_tablet_manager.cc:1403] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.088s user 0.002s sys 0.000s
I20251028 02:59:54.927281 8545 raft_consensus.cc:359] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.927424 8545 raft_consensus.cc:385] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.927465 8545 raft_consensus.cc:740] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.927596 8545 consensus_queue.cc:260] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.927670 8550 ts_tablet_manager.cc:1929] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.927735 8550 log.cc:1199] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c083ca984a7347a6b5bb0f3a949a5f75
I20251028 02:59:54.927884 8545 ts_tablet_manager.cc:1434] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:54.927956 8550 ts_tablet_manager.cc:1950] T c083ca984a7347a6b5bb0f3a949a5f75 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.928092 8545 tablet_bootstrap.cc:492] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.929124 8545 tablet_bootstrap.cc:654] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.929925 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84c3602bee3b4334a2a30daec986791d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.932466 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4902bc3398754802a5c571036cd19fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.933418 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet c083ca984a7347a6b5bb0f3a949a5f75 (table default.Table_d1b772f7ad114598bf453e2265c5e436 [id=73579e6e9da94fa699b3cb5bb5cec89e]) successfully deleted
W20251028 02:59:54.933868 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a3096fde2ac94ab197ce5e659c4d2be0 because the tablet was not found. No further retry: Not found: Tablet not found: a3096fde2ac94ab197ce5e659c4d2be0
I20251028 02:59:54.934051 8550 tablet_replica.cc:333] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.934183 8550 raft_consensus.cc:2243] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.934286 8550 raft_consensus.cc:2272] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.934706 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet e6f9b707ec0345249c3efec1c24a956e with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef353f5e715452eae1f1c191fe79d1d) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.934756 8562 raft_consensus.cc:493] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.934849 8562 raft_consensus.cc:515] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.934968 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 1094f4d2872449599e3eaf286def4f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.935195 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240b7da6f46c4c5ab8087ab1dd6275c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.936491 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de38b2d54cec4f4e892047f4650f611b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.936658 8552 tablet_replica.cc:333] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.936789 8552 raft_consensus.cc:2243] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.935204 8562 leader_election.cc:290] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.936898 8552 raft_consensus.cc:2272] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.937431 8552 ts_tablet_manager.cc:1916] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.937549 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaaec176a3884a79a55bb74397a7dc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.938098 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de38b2d54cec4f4e892047f4650f611b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:54.938230 8417 raft_consensus.cc:2468] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
W20251028 02:59:54.938310 8083 leader_election.cc:343] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:54.939913 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b already in progress: creating tablet
I20251028 02:59:54.940876 8550 ts_tablet_manager.cc:1916] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.942508 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 6f6f58ddaa6940248065a05b75996112 (DEFAULT_TABLE table=default.Table_8924bd5725814f70bc3834b45c618074 [id=b7ed75132db44133a09d9c0486be4706]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.942817 8083 leader_election.cc:304] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 02:59:54.943059 8562 raft_consensus.cc:2804] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:54.943111 8562 raft_consensus.cc:493] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:54.943148 8562 raft_consensus.cc:3060] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.943318 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f6f58ddaa6940248065a05b75996112. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.943359 8546 tablet_bootstrap.cc:492] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.943442 8546 ts_tablet_manager.cc:1403] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20251028 02:59:54.943998 8546 raft_consensus.cc:359] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.944042 8562 raft_consensus.cc:515] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.944108 8546 raft_consensus.cc:385] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.944139 8546 raft_consensus.cc:740] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.944260 8546 consensus_queue.cc:260] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.944392 8562 leader_election.cc:290] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.944600 8546 ts_tablet_manager.cc:1434] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.944726 8552 ts_tablet_manager.cc:1929] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.944783 8552 log.cc:1199] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/e6f9b707ec0345249c3efec1c24a956e
I20251028 02:59:54.944801 8546 tablet_bootstrap.cc:492] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.944981 8552 ts_tablet_manager.cc:1950] T e6f9b707ec0345249c3efec1c24a956e P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.945806 8546 tablet_bootstrap.cc:654] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.945969 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 6f6f58ddaa6940248065a05b75996112 (DEFAULT_TABLE table=default.Table_8924bd5725814f70bc3834b45c618074 [id=b7ed75132db44133a09d9c0486be4706]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.946228 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f6f58ddaa6940248065a05b75996112. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.947536 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de38b2d54cec4f4e892047f4650f611b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 02:59:54.947870 8083 leader_election.cc:343] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.947993 8562 raft_consensus.cc:493] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.948079 8562 raft_consensus.cc:515] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.948135 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de38b2d54cec4f4e892047f4650f611b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:54.948242 8417 raft_consensus.cc:3060] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.948410 8562 leader_election.cc:290] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.948909 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2ab414dbe9144aef8fd72a7f5a95577d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.949023 8283 raft_consensus.cc:2468] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:54.949288 8417 raft_consensus.cc:2468] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:54.949383 8083 leader_election.cc:304] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:54.949581 8333 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:54.949664 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet cdd92cd8a5194e22a91e59aaa4793135 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:54.949839 8083 leader_election.cc:304] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
W20251028 02:59:54.950057 8083 leader_election.cc:343] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 2ab414dbe9144aef8fd72a7f5a95577d
I20251028 02:59:54.950205 8656 raft_consensus.cc:2804] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:54.950249 8656 raft_consensus.cc:493] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:54.950289 8656 raft_consensus.cc:3060] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.950735 8550 ts_tablet_manager.cc:1929] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.950800 8550 log.cc:1199] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/e6f9b707ec0345249c3efec1c24a956e
I20251028 02:59:54.951026 8550 ts_tablet_manager.cc:1950] T e6f9b707ec0345249c3efec1c24a956e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.951102 8656 raft_consensus.cc:515] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.951455 8656 leader_election.cc:290] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.951547 8656 raft_consensus.cc:2804] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:54.951598 8656 raft_consensus.cc:697] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:54.951709 8656 consensus_queue.cc:237] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.952538 8550 tablet_replica.cc:333] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.952759 8550 raft_consensus.cc:2243] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.952943 8550 raft_consensus.cc:2272] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.953711 8550 ts_tablet_manager.cc:1916] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.956290 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet a198db14385543d99afb1881b0fe2409 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:54.957173 8550 ts_tablet_manager.cc:1929] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.957263 8550 log.cc:1199] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/69f1f6895a604b858b28a3e0241245de
I20251028 02:59:54.957253 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_053b1ca3666d4641b101c8c10f846980" } new_table_name: "default.Table_9bfa5c6f8a5340c1a3df4dfde6854803" modify_external_catalogs: true
I20251028 02:59:54.958174 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_38f86582ea6e4e5fb73330ab8f1d4c25" } modify_external_catalogs: true
I20251028 02:59:54.958257 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_38f86582ea6e4e5fb73330ab8f1d4c25" } modify_external_catalogs: true
W20251028 02:59:54.959492 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 81b521c9f57844bb9873f6b26b5cf661 consensus state has no leader
I20251028 02:59:54.959674 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_eb695c5a00aa482d958643fb13e4abaf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:54.959262 7966 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a] alter complete (version 1)
W20251028 02:59:54.959986 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb695c5a00aa482d958643fb13e4abaf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:54.960124 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_88e852b7e3584db7bfcfd5175978ade6" } new_table_name: "default.Table_11e7aae428ad4fc78c12c02f08986f46" modify_external_catalogs: true
I20251028 02:59:54.966774 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9a290b8114844aa89b33c75fffc8e1e7
I20251028 02:59:54.967665 8551 ts_tablet_manager.cc:1929] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:54.967797 8551 log.cc:1199] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/99c37c774ae2458989377a8bac94f4e8
I20251028 02:59:54.968008 8546 tablet_bootstrap.cc:492] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:54.968111 8546 ts_tablet_manager.cc:1403] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20251028 02:59:54.968598 8551 ts_tablet_manager.cc:1950] T 99c37c774ae2458989377a8bac94f4e8 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:54.968660 8546 raft_consensus.cc:359] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.968770 8546 raft_consensus.cc:385] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.968818 8546 raft_consensus.cc:740] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.968955 8546 consensus_queue.cc:260] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:54.969355 8544 tablet_bootstrap.cc:492] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:54.969448 8544 ts_tablet_manager.cc:1403] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.000s
I20251028 02:59:54.969915 8544 raft_consensus.cc:359] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.970016 8544 raft_consensus.cc:385] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.970057 8544 raft_consensus.cc:740] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.970168 8544 consensus_queue.cc:260] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.970485 8544 ts_tablet_manager.cc:1434] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.970657 8544 tablet_bootstrap.cc:492] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:54.971122 8545 tablet_bootstrap.cc:492] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:54.971208 8545 ts_tablet_manager.cc:1403] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.043s user 0.000s sys 0.002s
I20251028 02:59:54.971757 8545 raft_consensus.cc:359] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.971875 8545 raft_consensus.cc:385] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:54.971910 8545 raft_consensus.cc:740] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:54.972050 8546 ts_tablet_manager.cc:1434] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:54.972030 8545 consensus_queue.cc:260] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.972249 8546 tablet_bootstrap.cc:492] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:54.972325 8545 ts_tablet_manager.cc:1434] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:54.972368 8552 tablet_replica.cc:333] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:54.972487 8552 raft_consensus.cc:2243] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.972514 8545 tablet_bootstrap.cc:492] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:54.972577 8552 raft_consensus.cc:2272] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.973527 8545 tablet_bootstrap.cc:654] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.974259 8546 tablet_bootstrap.cc:654] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.976433 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 02695b18936d4f928cd55f9b76b90b2e (DEFAULT_TABLE table=default.Table_af589bb054d049a8beaed83cc51499f5 [id=058a75d461a0497a9268f8edf3506ece]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.976591 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2ab414dbe9144aef8fd72a7f5a95577d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:54.976703 8283 raft_consensus.cc:3060] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.976722 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02695b18936d4f928cd55f9b76b90b2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.973739 8552 ts_tablet_manager.cc:1916] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:54.977727 8283 raft_consensus.cc:2468] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:54.977746 8116 tablet_service.cc:1505] Processing CreateTablet for tablet b993932fd567416984ad6d28f8cadad6 (DEFAULT_TABLE table=default.Table_f56d1ad60fea4b2f9425aa0f9280df5d [id=b0407c8f78a84245a954c67e888da3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.977978 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b993932fd567416984ad6d28f8cadad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.978308 8083 leader_election.cc:304] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:54.978505 8562 raft_consensus.cc:2804] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:54.978587 8562 raft_consensus.cc:697] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:54.978721 8562 consensus_queue.cc:237] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.981925 8469 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:54.981984 8125 tablet_service.cc:1505] Processing CreateTablet for tablet c03faadb312742a4aecac0325f3595b7 (DEFAULT_TABLE table=default.Table_7fa5056d9def461dab21615f8b7e8e82 [id=da02d283856342fdaba51b17ce586311]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.982229 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c03faadb312742a4aecac0325f3595b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.983695 8544 tablet_bootstrap.cc:654] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:54.984786 8552 ts_tablet_manager.cc:1929] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:54.984861 8552 log.cc:1199] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/1094f4d2872449599e3eaf286def4f18
I20251028 02:59:54.985067 8552 ts_tablet_manager.cc:1950] T 1094f4d2872449599e3eaf286def4f18 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:54.985531 8965 raft_consensus.cc:493] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.985615 8965 raft_consensus.cc:515] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.985992 8965 leader_election.cc:290] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:54.986382 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56c8969da9dc464d9b463205e3a27965" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:54.986528 8421 raft_consensus.cc:2393] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in this term.
I20251028 02:59:54.986637 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a290b8114844aa89b33c75fffc8e1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.987365 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a198db14385543d99afb1881b0fe2409
W20251028 02:59:54.987479 8083 leader_election.cc:343] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 2ab414dbe9144aef8fd72a7f5a95577d
I20251028 02:59:54.987640 7969 catalog_manager.cc:6719] ReplaceTablet: tablet a198db14385543d99afb1881b0fe2409 deleted and replaced by tablet 302e0ed455854eb2b661afc2f2b6921d
I20251028 02:59:54.988054 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 5059e1ed0b494788bfd571987160b0b4 (DEFAULT_TABLE table=default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.988185 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56c8969da9dc464d9b463205e3a27965" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:54.988328 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5059e1ed0b494788bfd571987160b0b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.988543 8218 leader_election.cc:304] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.988767 8965 raft_consensus.cc:3060] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:54.989706 8965 raft_consensus.cc:2749] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.990186 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 8bca9ca8db1b491f9e6f7a564dc24ac1 (DEFAULT_TABLE table=default.Table_4d8bf6511df640c582ce134a8c146266 [id=472a765337f346e9bbfb73717b8c9074]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.990254 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 3f04c867f0ce4ab99f980cca063abc30 (DEFAULT_TABLE table=default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.990418 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bca9ca8db1b491f9e6f7a564dc24ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.990254 8123 tablet_service.cc:1505] Processing CreateTablet for tablet a76e16d10a174adda26b8ddd49ee528c (DEFAULT_TABLE table=default.Table_27f606e115954fd59700c86dabe70cae [id=400620abf0584fadbb666f17e41477b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.990578 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f04c867f0ce4ab99f980cca063abc30. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.990772 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a76e16d10a174adda26b8ddd49ee528c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.992686 9190 raft_consensus.cc:493] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:54.992781 9190 raft_consensus.cc:515] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:54.993093 8551 tablet_replica.cc:333] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:54.993156 9190 leader_election.cc:290] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:54.993232 8551 raft_consensus.cc:2243] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.993328 8551 raft_consensus.cc:2272] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:54.993583 8550 ts_tablet_manager.cc:1950] T 69f1f6895a604b858b28a3e0241245de P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:54.993592 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d81b3fe6073d4380882057eb97907fbb" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:54.993897 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d81b3fe6073d4380882057eb97907fbb" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:54.993957 8356 leader_election.cc:343] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:54.994220 8354 leader_election.cc:343] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:54.994284 8354 leader_election.cc:304] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:54.994472 9190 raft_consensus.cc:2749] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:54.995981 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet a198db14385543d99afb1881b0fe2409 with delete_type TABLET_DATA_DELETED (replaced by tablet 302e0ed455854eb2b661afc2f2b6921d) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:54.996327 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 6f6f58ddaa6940248065a05b75996112 (DEFAULT_TABLE table=default.Table_8924bd5725814f70bc3834b45c618074 [id=b7ed75132db44133a09d9c0486be4706]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:54.996603 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f6f58ddaa6940248065a05b75996112. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:54.997581 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 69f1f6895a604b858b28a3e0241245de (table default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]) successfully deleted
I20251028 02:59:54.998068 9122 tablet.cc:1722] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:54.998163 8550 tablet_replica.cc:333] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:54.998306 8550 raft_consensus.cc:2243] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:54.998411 8550 raft_consensus.cc:2272] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.000419 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 13fb883843d141f09aa90e66f9b7bf1f (DEFAULT_TABLE table=default.Table_afba81072ed644d4aac9d04bc5902e63 [id=c9c59047d50a4419a9f16431bce05cd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.000674 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13fb883843d141f09aa90e66f9b7bf1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.003358 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.003605 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.004901 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 94cbad596db44060bac2992e9ed17ed8 (DEFAULT_TABLE table=default.Table_9f6c006d57c74ddd9b5297aa3d310af4 [id=8dbb87b579c746e99322d3a93825d3a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.005155 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94cbad596db44060bac2992e9ed17ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.007614 8546 tablet_bootstrap.cc:492] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.007730 8546 ts_tablet_manager.cc:1403] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20251028 02:59:55.008370 8257 tablet_service.cc:1505] Processing CreateTablet for tablet c03faadb312742a4aecac0325f3595b7 (DEFAULT_TABLE table=default.Table_7fa5056d9def461dab21615f8b7e8e82 [id=da02d283856342fdaba51b17ce586311]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.008347 8546 raft_consensus.cc:359] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.008455 8546 raft_consensus.cc:385] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.008483 8546 raft_consensus.cc:740] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.008598 8546 consensus_queue.cc:260] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.008662 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 7801d57550dc40c39f2d3cf7e11e3713 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.008901 8546 ts_tablet_manager.cc:1434] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.008939 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a290b8114844aa89b33c75fffc8e1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.009016 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a290b8114844aa89b33c75fffc8e1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.009119 8546 tablet_bootstrap.cc:492] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.009668 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 02695b18936d4f928cd55f9b76b90b2e (DEFAULT_TABLE table=default.Table_af589bb054d049a8beaed83cc51499f5 [id=058a75d461a0497a9268f8edf3506ece]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.009951 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02695b18936d4f928cd55f9b76b90b2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.008607 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c03faadb312742a4aecac0325f3595b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.011592 8546 tablet_bootstrap.cc:654] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.011909 8395 tablet_service.cc:1505] Processing CreateTablet for tablet c03faadb312742a4aecac0325f3595b7 (DEFAULT_TABLE table=default.Table_7fa5056d9def461dab21615f8b7e8e82 [id=da02d283856342fdaba51b17ce586311]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.012122 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c03faadb312742a4aecac0325f3595b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.015053 9122 tablet.cc:1722] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:55.015653 9048 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 02:59:55.017236 9122 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 02:59:55.017261 9048 tablet.cc:1722] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:55.017563 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet de38b2d54cec4f4e892047f4650f611b with delete_type TABLET_DATA_DELETED (replaced by tablet b93581f0eb7e499597c4e5d2f6c058aa) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.017903 8395 tablet_service.cc:1505] Processing CreateTablet for tablet b993932fd567416984ad6d28f8cadad6 (DEFAULT_TABLE table=default.Table_f56d1ad60fea4b2f9425aa0f9280df5d [id=b0407c8f78a84245a954c67e888da3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.018159 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b993932fd567416984ad6d28f8cadad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.018990 9023 tablet.cc:1722] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:55.020854 8393 tablet_service.cc:1552] Processing DeleteTablet for tablet a198db14385543d99afb1881b0fe2409 with delete_type TABLET_DATA_DELETED (replaced by tablet 302e0ed455854eb2b661afc2f2b6921d) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.020973 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 8bca9ca8db1b491f9e6f7a564dc24ac1 (DEFAULT_TABLE table=default.Table_4d8bf6511df640c582ce134a8c146266 [id=472a765337f346e9bbfb73717b8c9074]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.021231 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bca9ca8db1b491f9e6f7a564dc24ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.021322 8393 tablet_service.cc:1505] Processing CreateTablet for tablet a76e16d10a174adda26b8ddd49ee528c (DEFAULT_TABLE table=default.Table_27f606e115954fd59700c86dabe70cae [id=400620abf0584fadbb666f17e41477b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.021548 8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a76e16d10a174adda26b8ddd49ee528c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.021507 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 8bca9ca8db1b491f9e6f7a564dc24ac1 (DEFAULT_TABLE table=default.Table_4d8bf6511df640c582ce134a8c146266 [id=472a765337f346e9bbfb73717b8c9074]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.021621 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 9e4eec90a2fa412bb0954805f5b8d832 (DEFAULT_TABLE table=default.Table_41a9163bd20143ff921de1c3d023fcfd [id=a4f9578fb0c146a7af9769f101d57da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.021740 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bca9ca8db1b491f9e6f7a564dc24ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.021904 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e4eec90a2fa412bb0954805f5b8d832. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.024012 8253 tablet_service.cc:1505] Processing CreateTablet for tablet a76e16d10a174adda26b8ddd49ee528c (DEFAULT_TABLE table=default.Table_27f606e115954fd59700c86dabe70cae [id=400620abf0584fadbb666f17e41477b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.024261 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a76e16d10a174adda26b8ddd49ee528c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.024456 8254 tablet_service.cc:1505] Processing CreateTablet for tablet 9e4eec90a2fa412bb0954805f5b8d832 (DEFAULT_TABLE table=default.Table_41a9163bd20143ff921de1c3d023fcfd [id=a4f9578fb0c146a7af9769f101d57da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.024673 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e4eec90a2fa412bb0954805f5b8d832. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.024009 8255 tablet_service.cc:1552] Processing DeleteTablet for tablet a198db14385543d99afb1881b0fe2409 with delete_type TABLET_DATA_DELETED (replaced by tablet 302e0ed455854eb2b661afc2f2b6921d) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.026701 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 5059e1ed0b494788bfd571987160b0b4 (DEFAULT_TABLE table=default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.026731 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e474b4516d5444bbb4b48bcbbbbf7cc1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.026921 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5059e1ed0b494788bfd571987160b0b4. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:55.027081 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e474b4516d5444bbb4b48bcbbbbf7cc1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.027938 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5008c46b0dcb452497e6dbfd59c37b92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.028281 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5008c46b0dcb452497e6dbfd59c37b92 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.028571 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 5059e1ed0b494788bfd571987160b0b4 (DEFAULT_TABLE table=default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.028838 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5059e1ed0b494788bfd571987160b0b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.028977 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 50e6d41d1cf746aab2802b5c6b2d58b3 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.029003 8258 tablet_service.cc:1505] Processing CreateTablet for tablet b993932fd567416984ad6d28f8cadad6 (DEFAULT_TABLE table=default.Table_f56d1ad60fea4b2f9425aa0f9280df5d [id=b0407c8f78a84245a954c67e888da3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.029213 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b993932fd567416984ad6d28f8cadad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.029176 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_27f606e115954fd59700c86dabe70cae" } new_table_name: "default.Table_273147e41d9f4c43b46e54c10ff21e09" modify_external_catalogs: true
I20251028 02:59:55.032277 8546 tablet_bootstrap.cc:492] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.032362 8546 ts_tablet_manager.cc:1403] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20251028 02:59:55.032824 8546 raft_consensus.cc:359] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.032920 8546 raft_consensus.cc:385] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.032950 8546 raft_consensus.cc:740] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.033072 8546 consensus_queue.cc:260] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.033363 8546 ts_tablet_manager.cc:1434] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.033536 8546 tablet_bootstrap.cc:492] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.033658 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 3f04c867f0ce4ab99f980cca063abc30 (DEFAULT_TABLE table=default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.033715 9023 tablet.cc:1722] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 02:59:55.033908 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f04c867f0ce4ab99f980cca063abc30. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.034499 8546 tablet_bootstrap.cc:654] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.038530 8254 tablet_service.cc:1505] Processing CreateTablet for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.039109 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.039372 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 923e46aca8284966980ac5b062baf829 (DEFAULT_TABLE table=default.Table_916c70151f044ff3b34a8e5313675a4f [id=ed2bacfbb3f24d5490ad58320bb33656]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.039417 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 13fb883843d141f09aa90e66f9b7bf1f (DEFAULT_TABLE table=default.Table_afba81072ed644d4aac9d04bc5902e63 [id=c9c59047d50a4419a9f16431bce05cd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.039726 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 3f04c867f0ce4ab99f980cca063abc30 (DEFAULT_TABLE table=default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.039925 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 99c37c774ae2458989377a8bac94f4e8 (table default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]) successfully deleted
I20251028 02:59:55.039933 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 923e46aca8284966980ac5b062baf829 (DEFAULT_TABLE table=default.Table_916c70151f044ff3b34a8e5313675a4f [id=ed2bacfbb3f24d5490ad58320bb33656]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.040196 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 923e46aca8284966980ac5b062baf829. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.041311 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ab414dbe9144aef8fd72a7f5a95577d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.041390 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 923e46aca8284966980ac5b062baf829. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.041352 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 13fb883843d141f09aa90e66f9b7bf1f (DEFAULT_TABLE table=default.Table_afba81072ed644d4aac9d04bc5902e63 [id=c9c59047d50a4419a9f16431bce05cd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.041564 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13fb883843d141f09aa90e66f9b7bf1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.041563 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 94cbad596db44060bac2992e9ed17ed8 (DEFAULT_TABLE table=default.Table_9f6c006d57c74ddd9b5297aa3d310af4 [id=8dbb87b579c746e99322d3a93825d3a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.041791 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 02695b18936d4f928cd55f9b76b90b2e (DEFAULT_TABLE table=default.Table_af589bb054d049a8beaed83cc51499f5 [id=058a75d461a0497a9268f8edf3506ece]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.043498 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94cbad596db44060bac2992e9ed17ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.044019 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9860023c7b564342ba3145eafe352beb
W20251028 02:59:55.049221 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06302d2cb5554805b85916741ea10f0c consensus state has no leader
I20251028 02:59:55.049425 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 35e211a1a8c547baaee1199f7103bfad from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:55.049471 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a76e16d10a174adda26b8ddd49ee528c consensus state has no leader
I20251028 02:59:55.049710 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_096243ad06e04bfba7e8b97d066cdf9c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.050158 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_096243ad06e04bfba7e8b97d066cdf9c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.050876 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6c4478bf2de14c3c824d9ab70557b237"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.051195 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c4478bf2de14c3c824d9ab70557b237 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.051215 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e1f5688d9de84f4b93cb599116c876bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.051563 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1f5688d9de84f4b93cb599116c876bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.051591 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 9e4eec90a2fa412bb0954805f5b8d832 (DEFAULT_TABLE table=default.Table_41a9163bd20143ff921de1c3d023fcfd [id=a4f9578fb0c146a7af9769f101d57da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.051916 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e4eec90a2fa412bb0954805f5b8d832. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.052248 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a00976e88f0147c5b01560dba597ea82" } modify_external_catalogs: true
I20251028 02:59:55.052341 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a00976e88f0147c5b01560dba597ea82" } modify_external_catalogs: true
I20251028 02:59:55.053241 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 50e6d41d1cf746aab2802b5c6b2d58b3
I20251028 02:59:55.053596 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ba1740929d404081b7451d23bc910475" } modify_external_catalogs: true
I20251028 02:59:55.053673 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ba1740929d404081b7451d23bc910475" } modify_external_catalogs: true
I20251028 02:59:55.052322 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 94cbad596db44060bac2992e9ed17ed8 (DEFAULT_TABLE table=default.Table_9f6c006d57c74ddd9b5297aa3d310af4 [id=8dbb87b579c746e99322d3a93825d3a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.053894 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94cbad596db44060bac2992e9ed17ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.054797 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 9860023c7b564342ba3145eafe352beb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.055660 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 923e46aca8284966980ac5b062baf829 (DEFAULT_TABLE table=default.Table_916c70151f044ff3b34a8e5313675a4f [id=ed2bacfbb3f24d5490ad58320bb33656]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.055958 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 923e46aca8284966980ac5b062baf829. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.057843 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 9860023c7b564342ba3145eafe352beb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.058069 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 9860023c7b564342ba3145eafe352beb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.058230 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.058565 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.058573 8550 ts_tablet_manager.cc:1916] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.059460 8551 ts_tablet_manager.cc:1916] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.060663 8550 ts_tablet_manager.cc:1929] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.060747 8550 log.cc:1199] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/1094f4d2872449599e3eaf286def4f18
I20251028 02:59:55.060976 8550 ts_tablet_manager.cc:1950] T 1094f4d2872449599e3eaf286def4f18 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.061569 8550 tablet_replica.cc:333] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.061688 8551 ts_tablet_manager.cc:1929] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.061714 8550 raft_consensus.cc:2243] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.061756 8551 log.cc:1199] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/1f03c8b5dabe49ad8f7fe01c7bbf5d7b
I20251028 02:59:55.061895 8550 pending_rounds.cc:70] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:55.061959 8551 ts_tablet_manager.cc:1950] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.061951 8550 pending_rounds.cc:77] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597137834778624 op_type: NO_OP noop_request { }
I20251028 02:59:55.062019 8550 raft_consensus.cc:2889] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:55.062080 8550 raft_consensus.cc:2272] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.062573 8552 tablet_replica.cc:333] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.062605 8551 tablet_replica.cc:333] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.062712 8551 raft_consensus.cc:2243] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.062709 8552 raft_consensus.cc:2243] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.062809 8551 raft_consensus.cc:2272] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.062814 8552 raft_consensus.cc:2272] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:55.063446 9377 consensus_queue.cc:1175] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20251028 02:59:55.063488 8551 ts_tablet_manager.cc:1916] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.065264 8552 ts_tablet_manager.cc:1916] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.065513 8551 ts_tablet_manager.cc:1929] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.065595 8551 log.cc:1199] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/de38b2d54cec4f4e892047f4650f611b
I20251028 02:59:55.067194 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.067713 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.068020 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 1094f4d2872449599e3eaf286def4f18 (table default.Table_f0712855c8944a5bbae3fc9c069b155d [id=c22cf53127924afcbc55ea4a1fd8c04f]) successfully deleted
I20251028 02:59:55.068243 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b (table default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]) successfully deleted
W20251028 02:59:55.068356 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 69f1f6895a604b858b28a3e0241245de because the tablet was not found. No further retry: Not found: Tablet not found: 69f1f6895a604b858b28a3e0241245de
I20251028 02:59:55.068487 8550 ts_tablet_manager.cc:1916] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.071674 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02695b18936d4f928cd55f9b76b90b2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.071643 8546 tablet_bootstrap.cc:492] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.071656 9381 tablet.cc:1722] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:55.071797 8546 ts_tablet_manager.cc:1403] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.001s
I20251028 02:59:55.072403 9382 tablet.cc:1722] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:55.072669 8546 raft_consensus.cc:359] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.072798 8546 raft_consensus.cc:385] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.072834 8546 raft_consensus.cc:740] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.072968 8546 consensus_queue.cc:260] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.073287 8552 ts_tablet_manager.cc:1929] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.073339 8546 ts_tablet_manager.cc:1434] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:55.073523 8546 tablet_bootstrap.cc:492] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.074572 8550 ts_tablet_manager.cc:1929] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.074644 8550 log.cc:1199] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2ab414dbe9144aef8fd72a7f5a95577d
I20251028 02:59:55.074666 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13fb883843d141f09aa90e66f9b7bf1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.075033 8551 ts_tablet_manager.cc:1950] T de38b2d54cec4f4e892047f4650f611b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.075925 8551 tablet_replica.cc:333] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.076157 8546 tablet_bootstrap.cc:654] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.076131 8551 raft_consensus.cc:2243] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.076300 8551 raft_consensus.cc:2272] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.073351 8552 log.cc:1199] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/cdd92cd8a5194e22a91e59aaa4793135
I20251028 02:59:55.076695 8552 ts_tablet_manager.cc:1950] T cdd92cd8a5194e22a91e59aaa4793135 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.077113 8551 ts_tablet_manager.cc:1916] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.077338 8552 tablet_replica.cc:333] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.077380 8544 tablet_bootstrap.cc:492] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.077461 8544 ts_tablet_manager.cc:1403] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.107s user 0.002s sys 0.000s
I20251028 02:59:55.077477 8552 raft_consensus.cc:2243] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.077579 8552 raft_consensus.cc:2272] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.077914 8544 raft_consensus.cc:359] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.078016 8544 raft_consensus.cc:385] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.078032 8552 ts_tablet_manager.cc:1916] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.078047 8544 raft_consensus.cc:740] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.078331 8544 consensus_queue.cc:260] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.078532 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f04c867f0ce4ab99f980cca063abc30. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.078639 8544 ts_tablet_manager.cc:1434] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.078846 8544 tablet_bootstrap.cc:492] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.078634 8550 ts_tablet_manager.cc:1950] T 2ab414dbe9144aef8fd72a7f5a95577d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.079962 8544 tablet_bootstrap.cc:654] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.080950 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 83059074c365440b84e39cf7a5553eb2 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.081099 8550 tablet_replica.cc:333] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.081190 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet cdd92cd8a5194e22a91e59aaa4793135 (table default.Table_54bb44e4b3aa4babb80308231391809c [id=2bd44c4bfd2741c7bec6f973ffc7682d]) successfully deleted
I20251028 02:59:55.081228 8550 raft_consensus.cc:2243] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.081357 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet de38b2d54cec4f4e892047f4650f611b (table default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]) successfully deleted
I20251028 02:59:55.081374 8550 pending_rounds.cc:70] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:55.081405 8550 pending_rounds.cc:77] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597136681713664 op_type: NO_OP noop_request { }
I20251028 02:59:55.081456 8550 raft_consensus.cc:2889] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:55.081506 8550 raft_consensus.cc:2272] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.082198 8550 ts_tablet_manager.cc:1916] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.084811 8552 ts_tablet_manager.cc:1929] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.084901 8552 log.cc:1199] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/7801d57550dc40c39f2d3cf7e11e3713
I20251028 02:59:55.085155 8552 ts_tablet_manager.cc:1950] T 7801d57550dc40c39f2d3cf7e11e3713 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.085595 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 7801d57550dc40c39f2d3cf7e11e3713 (table default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]) successfully deleted
I20251028 02:59:55.085677 8544 tablet_bootstrap.cc:492] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.085767 8544 ts_tablet_manager.cc:1403] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:55.086246 8544 raft_consensus.cc:359] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.086366 8544 raft_consensus.cc:385] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.086405 8544 raft_consensus.cc:740] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.086563 8544 consensus_queue.cc:260] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.086915 8544 ts_tablet_manager.cc:1434] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.086930 8545 tablet_bootstrap.cc:492] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.087018 8545 ts_tablet_manager.cc:1403] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.115s user 0.002s sys 0.000s
I20251028 02:59:55.087105 8544 tablet_bootstrap.cc:492] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.087535 8545 raft_consensus.cc:359] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.087651 8545 raft_consensus.cc:385] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.087685 8545 raft_consensus.cc:740] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.087801 8545 consensus_queue.cc:260] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.088081 8545 ts_tablet_manager.cc:1434] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.088137 8544 tablet_bootstrap.cc:654] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.088272 8545 tablet_bootstrap.cc:492] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.088681 8550 ts_tablet_manager.cc:1929] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.088768 8550 log.cc:1199] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/cdd92cd8a5194e22a91e59aaa4793135
I20251028 02:59:55.089054 8550 ts_tablet_manager.cc:1950] T cdd92cd8a5194e22a91e59aaa4793135 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.089043 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 1094f4d2872449599e3eaf286def4f18 (table default.Table_f0712855c8944a5bbae3fc9c069b155d [id=c22cf53127924afcbc55ea4a1fd8c04f]) successfully deleted
I20251028 02:59:55.089110 8546 tablet_bootstrap.cc:492] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.089188 8546 ts_tablet_manager.cc:1403] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.001s
I20251028 02:59:55.089394 8545 tablet_bootstrap.cc:654] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.089635 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2ab414dbe9144aef8fd72a7f5a95577d (table default.Table_44487777ef26464c9cbbb195199247a0 [id=f04d44307e3c4c07a2403477d11e9fcf]) successfully deleted
I20251028 02:59:55.089695 8546 raft_consensus.cc:359] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.089840 8546 raft_consensus.cc:385] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.089890 8546 raft_consensus.cc:740] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
W20251028 02:59:55.090021 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 9a290b8114844aa89b33c75fffc8e1e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a290b8114844aa89b33c75fffc8e1e7 already in progress: creating tablet
I20251028 02:59:55.090041 8546 consensus_queue.cc:260] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.090296 8552 tablet_replica.cc:333] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.090433 8552 raft_consensus.cc:2243] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.093621 8546 ts_tablet_manager.cc:1434] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:55.093855 8546 tablet_bootstrap.cc:492] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.093907 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 50e6d41d1cf746aab2802b5c6b2d58b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.094273 8551 ts_tablet_manager.cc:1929] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.094604 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 35e211a1a8c547baaee1199f7103bfad
I20251028 02:59:55.094612 8551 log.cc:1199] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/d81b3fe6073d4380882057eb97907fbb
I20251028 02:59:55.094921 8546 tablet_bootstrap.cc:654] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.094924 7966 catalog_manager.cc:6719] ReplaceTablet: tablet 35e211a1a8c547baaee1199f7103bfad deleted and replaced by tablet 51198eb794764f7fbfda806fc256bcca
I20251028 02:59:55.095484 8551 ts_tablet_manager.cc:1950] T d81b3fe6073d4380882057eb97907fbb P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.096611 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 35e211a1a8c547baaee1199f7103bfad with delete_type TABLET_DATA_DELETED (replaced by tablet 51198eb794764f7fbfda806fc256bcca) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.097378 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 35e211a1a8c547baaee1199f7103bfad with delete_type TABLET_DATA_DELETED (replaced by tablet 51198eb794764f7fbfda806fc256bcca) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.097571 8552 raft_consensus.cc:2272] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.097771 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet d81b3fe6073d4380882057eb97907fbb (table default.Table_916c70151f044ff3b34a8e5313675a4f [id=ed2bacfbb3f24d5490ad58320bb33656]) successfully deleted
I20251028 02:59:55.098120 8552 ts_tablet_manager.cc:1916] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.098152 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 35e211a1a8c547baaee1199f7103bfad with delete_type TABLET_DATA_DELETED (replaced by tablet 51198eb794764f7fbfda806fc256bcca) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:55.098212 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 9a290b8114844aa89b33c75fffc8e1e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a290b8114844aa89b33c75fffc8e1e7 already in progress: creating tablet
I20251028 02:59:55.098841 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet cdd92cd8a5194e22a91e59aaa4793135 (table default.Table_54bb44e4b3aa4babb80308231391809c [id=2bd44c4bfd2741c7bec6f973ffc7682d]) successfully deleted
W20251028 02:59:55.099099 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a198db14385543d99afb1881b0fe2409 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a198db14385543d99afb1881b0fe2409 already in progress: creating tablet
I20251028 02:59:55.099326 8551 tablet_replica.cc:333] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.100551 8552 ts_tablet_manager.cc:1929] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.100618 8552 log.cc:1199] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/de38b2d54cec4f4e892047f4650f611b
I20251028 02:59:55.100818 8552 ts_tablet_manager.cc:1950] T de38b2d54cec4f4e892047f4650f611b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.101423 8552 tablet_replica.cc:333] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.101445 8551 raft_consensus.cc:2243] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.101532 8552 raft_consensus.cc:2243] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.101575 8551 raft_consensus.cc:2272] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.101630 8552 raft_consensus.cc:2272] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.101881 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet de38b2d54cec4f4e892047f4650f611b (table default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]) successfully deleted
I20251028 02:59:55.101969 8551 ts_tablet_manager.cc:1916] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:55.102185 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a198db14385543d99afb1881b0fe2409 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a198db14385543d99afb1881b0fe2409 already in progress: creating tablet
I20251028 02:59:55.102474 8550 tablet_replica.cc:333] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.102741 8550 raft_consensus.cc:2243] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.102982 8550 raft_consensus.cc:2272] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.103650 8550 ts_tablet_manager.cc:1916] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.104234 9380 tablet.cc:1722] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:55.108400 8545 tablet_bootstrap.cc:492] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.108505 8545 ts_tablet_manager.cc:1403] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251028 02:59:55.109064 8545 raft_consensus.cc:359] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.109177 8545 raft_consensus.cc:385] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.109212 8545 raft_consensus.cc:740] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.109329 8545 consensus_queue.cc:260] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.109601 8545 ts_tablet_manager.cc:1434] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.109820 8545 tablet_bootstrap.cc:492] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.110879 8545 tablet_bootstrap.cc:654] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.110917 8550 ts_tablet_manager.cc:1929] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.111074 8550 log.cc:1199] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ba35c8fe21f140cda8a4b1ad7568ec75
I20251028 02:59:55.111454 8550 ts_tablet_manager.cc:1950] T ba35c8fe21f140cda8a4b1ad7568ec75 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.112152 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet 874b6660d86b48ebad03ecc54588da28 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.112850 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet ba35c8fe21f140cda8a4b1ad7568ec75 (table default.Table_2ad18e95c3a14e039a63bebaa46268fa [id=abb728d9aa374dce81ffa90585353fa7]) successfully deleted
I20251028 02:59:55.113246 8550 tablet_replica.cc:333] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.113507 8550 raft_consensus.cc:2243] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.114871 8550 raft_consensus.cc:2272] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.115689 8550 ts_tablet_manager.cc:1916] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.115829 8552 ts_tablet_manager.cc:1916] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.118265 8552 ts_tablet_manager.cc:1929] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.118353 8552 log.cc:1199] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/2ab414dbe9144aef8fd72a7f5a95577d
I20251028 02:59:55.119125 8551 ts_tablet_manager.cc:1929] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.119216 8551 log.cc:1199] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9860023c7b564342ba3145eafe352beb
I20251028 02:59:55.119474 8551 ts_tablet_manager.cc:1950] T 9860023c7b564342ba3145eafe352beb P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.120251 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9860023c7b564342ba3145eafe352beb (table default.Table_38f86582ea6e4e5fb73330ab8f1d4c25 [id=b072a25c1a8b4f72ba778ac32c066e71]) successfully deleted
I20251028 02:59:55.120465 8550 ts_tablet_manager.cc:1929] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.120690 8550 log.cc:1199] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9f1c1e14665c4712bffa439ab98b2a6e
I20251028 02:59:55.121209 8550 ts_tablet_manager.cc:1950] T 9f1c1e14665c4712bffa439ab98b2a6e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.122515 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9f1c1e14665c4712bffa439ab98b2a6e (table default.Table_6aba0ca2a3074aac8a0bf7f2da936e1a [id=3eb0b70129be40e598874d492735ba51]) successfully deleted
I20251028 02:59:55.123076 8545 tablet_bootstrap.cc:492] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.123183 8545 ts_tablet_manager.cc:1403] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20251028 02:59:55.123754 8545 raft_consensus.cc:359] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.123863 8545 raft_consensus.cc:385] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.123900 8545 raft_consensus.cc:740] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.124042 8545 consensus_queue.cc:260] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.124094 9366 consensus_queue.cc:1048] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:55.124361 8545 ts_tablet_manager.cc:1434] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.124572 8545 tablet_bootstrap.cc:492] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:55.125218 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet d59acfb0e12643ebbb406dbda7bef29b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d59acfb0e12643ebbb406dbda7bef29b already in progress: creating tablet
I20251028 02:59:55.125658 8545 tablet_bootstrap.cc:654] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.125653 8550 tablet_replica.cc:333] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.125965 8550 raft_consensus.cc:2243] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.126223 8550 raft_consensus.cc:2272] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.127409 8552 ts_tablet_manager.cc:1950] T 2ab414dbe9144aef8fd72a7f5a95577d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.127607 8550 ts_tablet_manager.cc:1916] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.127990 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3b43da9580244346ae512a7bc9b506d3
I20251028 02:59:55.128207 8552 tablet_replica.cc:333] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.128338 8552 raft_consensus.cc:2243] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.128445 8552 raft_consensus.cc:2272] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.128605 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 2ab414dbe9144aef8fd72a7f5a95577d (table default.Table_44487777ef26464c9cbbb195199247a0 [id=f04d44307e3c4c07a2403477d11e9fcf]) successfully deleted
I20251028 02:59:55.128762 8551 tablet_replica.cc:333] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.128808 8552 ts_tablet_manager.cc:1916] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.128928 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b43da9580244346ae512a7bc9b506d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.129091 8551 raft_consensus.cc:2243] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.129212 8551 raft_consensus.cc:2272] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.130126 8551 ts_tablet_manager.cc:1916] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.130225 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b43da9580244346ae512a7bc9b506d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.130908 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b43da9580244346ae512a7bc9b506d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.133210 8546 tablet_bootstrap.cc:492] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.133293 8546 ts_tablet_manager.cc:1403] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20251028 02:59:55.133742 8546 raft_consensus.cc:359] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.133836 8546 raft_consensus.cc:385] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.133863 8546 raft_consensus.cc:740] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.133977 8546 consensus_queue.cc:260] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.134065 8550 ts_tablet_manager.cc:1929] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.134196 8550 log.cc:1199] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/1f03c8b5dabe49ad8f7fe01c7bbf5d7b
I20251028 02:59:55.134224 8546 ts_tablet_manager.cc:1434] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.134409 8546 tablet_bootstrap.cc:492] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.134600 8550 ts_tablet_manager.cc:1950] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.135766 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b (table default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]) successfully deleted
I20251028 02:59:55.136196 8550 tablet_replica.cc:333] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.136484 8550 raft_consensus.cc:2243] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.136708 8546 tablet_bootstrap.cc:654] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.136896 8550 raft_consensus.cc:2272] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.137641 8550 ts_tablet_manager.cc:1916] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.137732 8552 ts_tablet_manager.cc:1929] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.137881 8552 log.cc:1199] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9860023c7b564342ba3145eafe352beb
I20251028 02:59:55.138154 8552 ts_tablet_manager.cc:1950] T 9860023c7b564342ba3145eafe352beb P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.138862 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9860023c7b564342ba3145eafe352beb (table default.Table_38f86582ea6e4e5fb73330ab8f1d4c25 [id=b072a25c1a8b4f72ba778ac32c066e71]) successfully deleted
I20251028 02:59:55.141360 8551 ts_tablet_manager.cc:1929] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.141453 8551 log.cc:1199] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/50e6d41d1cf746aab2802b5c6b2d58b3
I20251028 02:59:55.141636 8552 tablet_replica.cc:333] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.141703 8551 ts_tablet_manager.cc:1950] T 50e6d41d1cf746aab2802b5c6b2d58b3 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.141894 8552 raft_consensus.cc:2243] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.142079 8552 raft_consensus.cc:2272] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.142413 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (table default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]) successfully deleted
I20251028 02:59:55.142670 8549 raft_consensus.cc:493] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.142771 8552 ts_tablet_manager.cc:1916] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.142757 8549 raft_consensus.cc:515] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.143187 8549 leader_election.cc:290] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.143249 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 506396e55fc242ada23d66088984f0ae
I20251028 02:59:55.143571 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12dce4bb3d4b43608d7da9350b14e1d0" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:55.143705 8147 raft_consensus.cc:2468] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:55.144016 8354 leader_election.cc:304] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.144071 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 506396e55fc242ada23d66088984f0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.144156 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12dce4bb3d4b43608d7da9350b14e1d0" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.144233 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 506396e55fc242ada23d66088984f0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.144342 8549 raft_consensus.cc:2804] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.144400 8549 raft_consensus.cc:493] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.144415 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 506396e55fc242ada23d66088984f0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.144451 8549 raft_consensus.cc:3060] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.144681 8550 ts_tablet_manager.cc:1929] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.144790 8550 log.cc:1199] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/99c37c774ae2458989377a8bac94f4e8
W20251028 02:59:55.144886 8356 leader_election.cc:343] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.145251 8550 ts_tablet_manager.cc:1950] T 99c37c774ae2458989377a8bac94f4e8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.145489 8549 raft_consensus.cc:515] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.145864 8549 leader_election.cc:290] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.146212 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12dce4bb3d4b43608d7da9350b14e1d0" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:55.146354 8147 raft_consensus.cc:3060] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.146535 8545 tablet_bootstrap.cc:492] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.146623 8545 ts_tablet_manager.cc:1403] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20251028 02:59:55.146669 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 99c37c774ae2458989377a8bac94f4e8 (table default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]) successfully deleted
I20251028 02:59:55.146950 8550 tablet_replica.cc:333] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.147154 8545 raft_consensus.cc:359] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.147192 8550 raft_consensus.cc:2243] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.147248 8545 raft_consensus.cc:385] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.147284 8545 raft_consensus.cc:740] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.147482 8550 pending_rounds.cc:70] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:55.147471 8545 consensus_queue.cc:260] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.147544 8550 pending_rounds.cc:77] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597137723580416 op_type: NO_OP noop_request { }
I20251028 02:59:55.147581 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12dce4bb3d4b43608d7da9350b14e1d0" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.147615 8550 raft_consensus.cc:2889] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:55.147722 8550 raft_consensus.cc:2272] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.147809 8545 ts_tablet_manager.cc:1434] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20251028 02:59:55.148191 8356 leader_election.cc:343] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.147488 8147 raft_consensus.cc:2468] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:55.148540 8556 raft_consensus.cc:493] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.148618 8556 raft_consensus.cc:515] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.148816 8354 leader_election.cc:304] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 02:59:55.148871 8544 tablet_bootstrap.cc:492] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.148953 8544 ts_tablet_manager.cc:1403] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.062s user 0.002s sys 0.000s
I20251028 02:59:55.149006 8556 leader_election.cc:290] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.149140 8549 raft_consensus.cc:2804] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.149228 8549 raft_consensus.cc:697] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:55.149354 8549 consensus_queue.cc:237] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.149418 8544 raft_consensus.cc:359] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.149509 8544 raft_consensus.cc:385] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.149542 8544 raft_consensus.cc:740] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.149652 8544 consensus_queue.cc:260] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.149960 8544 ts_tablet_manager.cc:1434] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.150153 8544 tablet_bootstrap.cc:492] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.151376 8196 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:55.151520 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "617ca0f545824d14ba7270619f121bcc" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.151628 8417 raft_consensus.cc:2468] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:55.152206 8220 leader_election.cc:304] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:55.152386 8556 raft_consensus.cc:2804] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.152434 8556 raft_consensus.cc:493] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.152472 8556 raft_consensus.cc:3060] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.152824 8545 tablet_bootstrap.cc:492] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.153529 8556 raft_consensus.cc:515] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.153911 8556 leader_election.cc:290] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.154253 8545 tablet_bootstrap.cc:654] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.154446 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "617ca0f545824d14ba7270619f121bcc" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:55.154822 8546 tablet_bootstrap.cc:492] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.154911 8546 ts_tablet_manager.cc:1403] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.001s
I20251028 02:59:55.154981 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "617ca0f545824d14ba7270619f121bcc" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.155081 8417 raft_consensus.cc:3060] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.155437 8546 raft_consensus.cc:359] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.155521 8546 raft_consensus.cc:385] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.155550 8546 raft_consensus.cc:740] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.155656 8546 consensus_queue.cc:260] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:55.155876 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 35e211a1a8c547baaee1199f7103bfad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35e211a1a8c547baaee1199f7103bfad already in progress: creating tablet
I20251028 02:59:55.155977 8546 ts_tablet_manager.cc:1434] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:55.156036 8417 raft_consensus.cc:2468] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:55.156190 8546 tablet_bootstrap.cc:492] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:55.156312 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 3b43da9580244346ae512a7bc9b506d3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b43da9580244346ae512a7bc9b506d3 already in progress: creating tablet
W20251028 02:59:55.156680 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 506396e55fc242ada23d66088984f0ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 506396e55fc242ada23d66088984f0ae already in progress: creating tablet
I20251028 02:59:55.157153 8546 tablet_bootstrap.cc:654] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.157517 8220 leader_election.cc:304] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:55.157733 8553 raft_consensus.cc:2804] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.157747 8545 tablet_bootstrap.cc:492] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.157824 8545 ts_tablet_manager.cc:1403] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20251028 02:59:55.157819 8553 raft_consensus.cc:697] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:55.157936 8553 consensus_queue.cc:237] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.158273 9366 raft_consensus.cc:493] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.158319 8545 raft_consensus.cc:359] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.158407 8545 raft_consensus.cc:385] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.158363 9366 raft_consensus.cc:515] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.158437 8545 raft_consensus.cc:740] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.158542 8545 consensus_queue.cc:260] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.158645 7967 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2] alter complete (version 1)
I20251028 02:59:55.158740 9366 leader_election.cc:290] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 02:59:55.158846 8218 leader_election.cc:343] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.159255 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ec69044a4c3e4f049f90be835cf73367" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.159375 8284 raft_consensus.cc:2468] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.159642 8083 leader_election.cc:304] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.159843 9366 raft_consensus.cc:2804] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.159898 9366 raft_consensus.cc:493] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.159937 9366 raft_consensus.cc:3060] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.160959 8545 ts_tablet_manager.cc:1434] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:55.161033 9366 raft_consensus.cc:515] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.161386 9366 leader_election.cc:290] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.161784 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ec69044a4c3e4f049f90be835cf73367" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.161880 8284 raft_consensus.cc:3060] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.162163 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "617ca0f545824d14ba7270619f121bcc" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:55.162500 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ec69044a4c3e4f049f90be835cf73367" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.162602 8417 raft_consensus.cc:2468] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.162674 8284 raft_consensus.cc:2468] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.162932 8544 tablet_bootstrap.cc:654] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.163046 8083 leader_election.cc:304] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
W20251028 02:59:55.163688 8218 leader_election.cc:343] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.163894 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ec69044a4c3e4f049f90be835cf73367" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.164006 8421 raft_consensus.cc:3060] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.164784 9366 raft_consensus.cc:2804] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.164880 9366 raft_consensus.cc:697] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:55.164913 8421 raft_consensus.cc:2468] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.165012 9366 consensus_queue.cc:237] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.168064 8545 tablet_bootstrap.cc:492] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.168890 8469 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
W20251028 02:59:55.169004 8083 consensus_peers.cc:597] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9860023c7b564342ba3145eafe352beb. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:55.169917 8562 consensus_queue.cc:1048] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:55.170964 8552 ts_tablet_manager.cc:1929] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.173687 8545 tablet_bootstrap.cc:654] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.175136 7969 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_8f6fdbd222f4456596107350e6c3c6e1 [id=4eb2743632814baab2e39bbdce700d6b] alter complete (version 5)
I20251028 02:59:55.175520 8552 log.cc:1199] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/50e6d41d1cf746aab2802b5c6b2d58b3
I20251028 02:59:55.175900 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 83059074c365440b84e39cf7a5553eb2
I20251028 02:59:55.175925 8552 ts_tablet_manager.cc:1950] T 50e6d41d1cf746aab2802b5c6b2d58b3 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.176188 7961 catalog_manager.cc:6719] ReplaceTablet: tablet 83059074c365440b84e39cf7a5553eb2 deleted and replaced by tablet c120a3d56f6b41c28c7f7dcd84359a60
I20251028 02:59:55.176339 8545 tablet_bootstrap.cc:492] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.176429 8545 ts_tablet_manager.cc:1403] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20251028 02:59:55.176823 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 874b6660d86b48ebad03ecc54588da28
I20251028 02:59:55.176934 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 83059074c365440b84e39cf7a5553eb2 with delete_type TABLET_DATA_DELETED (replaced by tablet c120a3d56f6b41c28c7f7dcd84359a60) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.177018 7962 catalog_manager.cc:6719] ReplaceTablet: tablet 874b6660d86b48ebad03ecc54588da28 deleted and replaced by tablet 1d3e422c7ae64b25960714ecc926046a
I20251028 02:59:55.177069 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ab414dbe9144aef8fd72a7f5a95577d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.177088 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b with delete_type TABLET_DATA_DELETED (replaced by tablet 55e72a7f8aa842049da7aeaaa7001a28) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.177165 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 69f1f6895a604b858b28a3e0241245de with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.177551 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (table default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]) successfully deleted
I20251028 02:59:55.177886 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 83059074c365440b84e39cf7a5553eb2 with delete_type TABLET_DATA_DELETED (replaced by tablet c120a3d56f6b41c28c7f7dcd84359a60) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.178269 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b with delete_type TABLET_DATA_DELETED (replaced by tablet 55e72a7f8aa842049da7aeaaa7001a28) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.178601 8545 raft_consensus.cc:359] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.178750 8545 raft_consensus.cc:385] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.178789 8545 raft_consensus.cc:740] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.179057 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 874b6660d86b48ebad03ecc54588da28 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d3e422c7ae64b25960714ecc926046a) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.179327 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 874b6660d86b48ebad03ecc54588da28 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d3e422c7ae64b25960714ecc926046a) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:55.179374 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 35e211a1a8c547baaee1199f7103bfad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35e211a1a8c547baaee1199f7103bfad already in progress: creating tablet
I20251028 02:59:55.179605 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 83059074c365440b84e39cf7a5553eb2 with delete_type TABLET_DATA_DELETED (replaced by tablet c120a3d56f6b41c28c7f7dcd84359a60) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:55.179809 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 3b43da9580244346ae512a7bc9b506d3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b43da9580244346ae512a7bc9b506d3 already in progress: creating tablet
W20251028 02:59:55.180167 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 506396e55fc242ada23d66088984f0ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 506396e55fc242ada23d66088984f0ae already in progress: creating tablet
I20251028 02:59:55.178974 8545 consensus_queue.cc:260] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.181272 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 56c8969da9dc464d9b463205e3a27965 (table default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]). Expected version 1 got 0
I20251028 02:59:55.183861 8545 ts_tablet_manager.cc:1434] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20251028 02:59:55.183907 8552 tablet_replica.cc:333] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.184044 8552 raft_consensus.cc:2243] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.184115 8546 tablet_bootstrap.cc:492] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.184149 8552 raft_consensus.cc:2272] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.184191 8546 ts_tablet_manager.cc:1403] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.001s
I20251028 02:59:55.184279 8545 tablet_bootstrap.cc:492] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.184633 8546 raft_consensus.cc:359] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.184725 8546 raft_consensus.cc:385] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.184752 8546 raft_consensus.cc:740] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.184859 8546 consensus_queue.cc:260] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.185104 8546 ts_tablet_manager.cc:1434] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.185253 8546 tablet_bootstrap.cc:492] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.185379 8545 tablet_bootstrap.cc:654] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.186161 8546 tablet_bootstrap.cc:654] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.187428 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 56c8969da9dc464d9b463205e3a27965 (table default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]). Expected version 1 got 0
I20251028 02:59:55.188057 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 874b6660d86b48ebad03ecc54588da28 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d3e422c7ae64b25960714ecc926046a) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.188899 8552 ts_tablet_manager.cc:1916] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.190003 8550 ts_tablet_manager.cc:1916] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:55.190294 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 83059074c365440b84e39cf7a5553eb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83059074c365440b84e39cf7a5553eb2 already in progress: creating tablet
W20251028 02:59:55.190727 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b because the tablet was not found. No further retry: Not found: Tablet not found: 1f03c8b5dabe49ad8f7fe01c7bbf5d7b
W20251028 02:59:55.191104 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 874b6660d86b48ebad03ecc54588da28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 874b6660d86b48ebad03ecc54588da28 already in progress: creating tablet
I20251028 02:59:55.191319 8333 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:55.192571 8552 ts_tablet_manager.cc:1929] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.195612 8545 tablet_bootstrap.cc:492] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.195741 8545 ts_tablet_manager.cc:1403] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20251028 02:59:55.195863 8552 log.cc:1199] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/69f1f6895a604b858b28a3e0241245de
I20251028 02:59:55.196058 8550 ts_tablet_manager.cc:1929] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.196166 8550 log.cc:1199] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/de38b2d54cec4f4e892047f4650f611b
I20251028 02:59:55.196368 8552 ts_tablet_manager.cc:1950] T 69f1f6895a604b858b28a3e0241245de P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.196346 8545 raft_consensus.cc:359] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.196638 8550 ts_tablet_manager.cc:1950] T de38b2d54cec4f4e892047f4650f611b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.196477 8545 raft_consensus.cc:385] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.197152 8545 raft_consensus.cc:740] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.197348 8545 consensus_queue.cc:260] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.197785 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet de38b2d54cec4f4e892047f4650f611b (table default.Table_1692c48a1cb243039125140ab7932f6b [id=852bae9da8c24f93898f6e648067b1e0]) successfully deleted
I20251028 02:59:55.198155 8550 tablet_replica.cc:333] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.198436 8550 raft_consensus.cc:2243] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.198715 8550 raft_consensus.cc:2272] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.199589 8550 ts_tablet_manager.cc:1916] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.200959 8544 tablet_bootstrap.cc:492] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.201061 8544 ts_tablet_manager.cc:1403] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.051s user 0.000s sys 0.002s
I20251028 02:59:55.201094 8546 tablet_bootstrap.cc:492] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.201170 8546 ts_tablet_manager.cc:1403] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20251028 02:59:55.201565 8544 raft_consensus.cc:359] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.201603 8546 raft_consensus.cc:359] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.201678 8544 raft_consensus.cc:385] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.201692 8546 raft_consensus.cc:385] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.201714 8544 raft_consensus.cc:740] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.201727 8546 raft_consensus.cc:740] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.201779 8545 ts_tablet_manager.cc:1434] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:55.201848 8546 consensus_queue.cc:260] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.201848 8544 consensus_queue.cc:260] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.202121 8544 ts_tablet_manager.cc:1434] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:55.202154 8546 ts_tablet_manager.cc:1434] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.202312 8546 tablet_bootstrap.cc:492] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.202312 8544 tablet_bootstrap.cc:492] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.204156 8544 tablet_bootstrap.cc:654] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.204682 8545 tablet_bootstrap.cc:492] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.205585 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 69f1f6895a604b858b28a3e0241245de (table default.Table_fc88d863fb634dc59631a752b9afe967 [id=0c82d04050b246bcb8ce902cff64ff1e]) successfully deleted
W20251028 02:59:55.205785 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 874b6660d86b48ebad03ecc54588da28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 874b6660d86b48ebad03ecc54588da28 already in progress: creating tablet
W20251028 02:59:55.205955 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 83059074c365440b84e39cf7a5553eb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83059074c365440b84e39cf7a5553eb2 already in progress: creating tablet
I20251028 02:59:55.206470 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_47e366fafa11436db14f8076fd1f1ba2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.206908 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47e366fafa11436db14f8076fd1f1ba2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.207850 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_41a9163bd20143ff921de1c3d023fcfd" } new_table_name: "default.Table_086b616472fc449eacfaa348042948e0" modify_external_catalogs: true
I20251028 02:59:55.208570 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_11e7aae428ad4fc78c12c02f08986f46" } new_table_name: "default.Table_2af68266ea4d4f288ecc63e064bb69db" modify_external_catalogs: true
I20251028 02:59:55.208720 8545 tablet_bootstrap.cc:654] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.209208 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4f7a6b08bb2b4cee90e54afd8f1b801a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.209506 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f7a6b08bb2b4cee90e54afd8f1b801a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.211906 8546 tablet_bootstrap.cc:654] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.212091 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_122813ac12be41ba91423e5309fa8eab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.212368 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 56c8969da9dc464d9b463205e3a27965 (table default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]). Expected version 1 got 0
W20251028 02:59:55.212393 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_122813ac12be41ba91423e5309fa8eab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.213603 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_9bfa5c6f8a5340c1a3df4dfde6854803 [id=7eb5bb61f05d4d79b43385b5e91c5b2e] alter complete (version 1)
I20251028 02:59:55.213821 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 6bd72bb34fe843079f8a4b828770e441 (DEFAULT_TABLE table=default.Table_a90dbf2ca45f4e3d8e9352c7eb8b3d99 [id=262048a24b0f47b0b3c0dee427389e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.214129 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bd72bb34fe843079f8a4b828770e441. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.214344 8125 tablet_service.cc:1505] Processing CreateTablet for tablet e95a45cb612447dd9bebd9f7a019d1f9 (DEFAULT_TABLE table=default.Table_eb695c5a00aa482d958643fb13e4abaf [id=7e087d7232254c0e87aabe25be6eb25d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.214393 8544 tablet_bootstrap.cc:492] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.214484 8544 ts_tablet_manager.cc:1403] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20251028 02:59:55.214550 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95a45cb612447dd9bebd9f7a019d1f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.214972 8544 raft_consensus.cc:359] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.215080 8544 raft_consensus.cc:385] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.215112 8544 raft_consensus.cc:740] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.215234 8544 consensus_queue.cc:260] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.215584 8544 ts_tablet_manager.cc:1434] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:55.215764 8544 tablet_bootstrap.cc:492] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.216441 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 81c2c40e4da242f8891687a6365d273c (DEFAULT_TABLE table=default.Table_48dcb7bf58c748a28999785ae72f248a [id=1cf0d713d3de40858825646f084e8505]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.216488 8126 tablet_service.cc:1505] Processing CreateTablet for tablet f28ca2cf41564792b49940aebc9b2c98 (DEFAULT_TABLE table=default.Table_5008c46b0dcb452497e6dbfd59c37b92 [id=a7962ddee0e841c0ad97786634c03a75]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.216717 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81c2c40e4da242f8891687a6365d273c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.216749 8544 tablet_bootstrap.cc:654] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.217607 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 1979906599754f7ba049c7a9b23bf26d (DEFAULT_TABLE table=default.Table_e474b4516d5444bbb4b48bcbbbbf7cc1 [id=9fcc70dd88724a28b404e9d4109015aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.217826 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1979906599754f7ba049c7a9b23bf26d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.219506 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f28ca2cf41564792b49940aebc9b2c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.219878 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 1b436561f0bf44e0b2c680ce30919318 (DEFAULT_TABLE table=default.Table_e3ed44682141426f856f581db3e481af [id=f34cf093333b4460b55dd2a474688449]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.220069 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b436561f0bf44e0b2c680ce30919318. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.220252 8544 tablet_bootstrap.cc:492] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.220333 8544 ts_tablet_manager.cc:1403] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20251028 02:59:55.220788 8544 raft_consensus.cc:359] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.220873 8544 raft_consensus.cc:385] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.220894 8544 raft_consensus.cc:740] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.220999 8544 consensus_queue.cc:260] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.221266 8544 ts_tablet_manager.cc:1434] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:55.221438 8544 tablet_bootstrap.cc:492] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.222507 8544 tablet_bootstrap.cc:654] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.223500 8544 tablet_bootstrap.cc:492] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.223574 8544 ts_tablet_manager.cc:1403] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20251028 02:59:55.223856 8545 tablet_bootstrap.cc:492] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.223945 8545 ts_tablet_manager.cc:1403] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20251028 02:59:55.224360 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 6bd72bb34fe843079f8a4b828770e441 (DEFAULT_TABLE table=default.Table_a90dbf2ca45f4e3d8e9352c7eb8b3d99 [id=262048a24b0f47b0b3c0dee427389e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.224480 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 1979906599754f7ba049c7a9b23bf26d (DEFAULT_TABLE table=default.Table_e474b4516d5444bbb4b48bcbbbbf7cc1 [id=9fcc70dd88724a28b404e9d4109015aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.224541 9190 raft_consensus.cc:493] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.224602 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bd72bb34fe843079f8a4b828770e441. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.224686 8544 raft_consensus.cc:359] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.224778 8544 raft_consensus.cc:385] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.224812 8544 raft_consensus.cc:740] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.224752 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 81c2c40e4da242f8891687a6365d273c (DEFAULT_TABLE table=default.Table_48dcb7bf58c748a28999785ae72f248a [id=1cf0d713d3de40858825646f084e8505]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.224889 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1979906599754f7ba049c7a9b23bf26d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.224956 8544 consensus_queue.cc:260] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.225107 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 1b436561f0bf44e0b2c680ce30919318 (DEFAULT_TABLE table=default.Table_e3ed44682141426f856f581db3e481af [id=f34cf093333b4460b55dd2a474688449]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.225217 8544 ts_tablet_manager.cc:1434] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 02:59:55.225313 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b436561f0bf44e0b2c680ce30919318. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.225375 8544 tablet_bootstrap.cc:492] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.226385 8544 tablet_bootstrap.cc:654] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.226913 8393 tablet_service.cc:1505] Processing CreateTablet for tablet f28ca2cf41564792b49940aebc9b2c98 (DEFAULT_TABLE table=default.Table_5008c46b0dcb452497e6dbfd59c37b92 [id=a7962ddee0e841c0ad97786634c03a75]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.227128 8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f28ca2cf41564792b49940aebc9b2c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.227236 8545 raft_consensus.cc:359] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.227353 8545 raft_consensus.cc:385] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.227386 8545 raft_consensus.cc:740] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.227504 8545 consensus_queue.cc:260] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.228503 8545 ts_tablet_manager.cc:1434] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 02:59:55.224604 9190 raft_consensus.cc:515] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.228668 8545 tablet_bootstrap.cc:492] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.228910 9190 leader_election.cc:290] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.224339 8394 tablet_service.cc:1505] Processing CreateTablet for tablet e95a45cb612447dd9bebd9f7a019d1f9 (DEFAULT_TABLE table=default.Table_eb695c5a00aa482d958643fb13e4abaf [id=7e087d7232254c0e87aabe25be6eb25d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.229128 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95a45cb612447dd9bebd9f7a019d1f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.229671 8545 tablet_bootstrap.cc:654] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.230408 8556 raft_consensus.cc:493] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.230463 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet ec69044a4c3e4f049f90be835cf73367 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.230502 8556 raft_consensus.cc:515] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.230602 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet d81b3fe6073d4380882057eb97907fbb with delete_type TABLET_DATA_DELETED (replaced by tablet 923e46aca8284966980ac5b062baf829) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.230904 8556 leader_election.cc:290] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.231290 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d81b3fe6073d4380882057eb97907fbb" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:55.231433 8147 raft_consensus.cc:2468] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:55.231724 8218 leader_election.cc:304] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.231894 8556 raft_consensus.cc:2804] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
W20251028 02:59:55.231909 8220 leader_election.cc:343] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: d81b3fe6073d4380882057eb97907fbb
I20251028 02:59:55.231971 8556 raft_consensus.cc:493] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.232012 8556 raft_consensus.cc:3060] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.232604 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81c2c40e4da242f8891687a6365d273c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.232957 8556 raft_consensus.cc:515] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.233291 8556 leader_election.cc:290] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.235236 8546 tablet_bootstrap.cc:492] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.235352 8546 ts_tablet_manager.cc:1403] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.002s
I20251028 02:59:55.235560 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 81c2c40e4da242f8891687a6365d273c (DEFAULT_TABLE table=default.Table_48dcb7bf58c748a28999785ae72f248a [id=1cf0d713d3de40858825646f084e8505]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.235713 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet d81b3fe6073d4380882057eb97907fbb with delete_type TABLET_DATA_DELETED (replaced by tablet 923e46aca8284966980ac5b062baf829) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.235783 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81c2c40e4da242f8891687a6365d273c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.235868 8546 raft_consensus.cc:359] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.235872 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b with delete_type TABLET_DATA_DELETED (replaced by tablet 55e72a7f8aa842049da7aeaaa7001a28) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.235962 8546 raft_consensus.cc:385] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.235993 8546 raft_consensus.cc:740] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.236120 8546 consensus_queue.cc:260] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.236392 8546 ts_tablet_manager.cc:1434] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.236626 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet ec69044a4c3e4f049f90be835cf73367 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.236634 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d6e8f311ccc94210b7a22755a7b4c21a" } modify_external_catalogs: true
I20251028 02:59:55.236748 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet 617ca0f545824d14ba7270619f121bcc with delete_type TABLET_DATA_DELETED (replaced by tablet 73c20437695a4043a86faa9c98091e5e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.236755 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d6e8f311ccc94210b7a22755a7b4c21a" } modify_external_catalogs: true
W20251028 02:59:55.236913 8220 leader_election.cc:343] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: d81b3fe6073d4380882057eb97907fbb
I20251028 02:59:55.237983 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 6bd72bb34fe843079f8a4b828770e441 (DEFAULT_TABLE table=default.Table_a90dbf2ca45f4e3d8e9352c7eb8b3d99 [id=262048a24b0f47b0b3c0dee427389e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.238215 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bd72bb34fe843079f8a4b828770e441. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.239045 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "19efc09c08b540a3b42c4107de5acf73" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:55.239332 8552 tablet_replica.cc:333] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.239480 8552 raft_consensus.cc:2243] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.239573 8552 raft_consensus.cc:2272] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.239727 8545 tablet_bootstrap.cc:492] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.239820 8545 ts_tablet_manager.cc:1403] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:55.239992 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7bfb43f76f8b46b292deee50a8f58280"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.240126 8552 ts_tablet_manager.cc:1916] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.240319 8545 raft_consensus.cc:359] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.240378 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 1979906599754f7ba049c7a9b23bf26d (DEFAULT_TABLE table=default.Table_e474b4516d5444bbb4b48bcbbbbf7cc1 [id=9fcc70dd88724a28b404e9d4109015aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.240419 8545 raft_consensus.cc:385] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251028 02:59:55.240401 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7bfb43f76f8b46b292deee50a8f58280 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.240451 8545 raft_consensus.cc:740] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.240571 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1979906599754f7ba049c7a9b23bf26d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.240571 8545 consensus_queue.cc:260] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.240865 8545 ts_tablet_manager.cc:1434] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.241029 8545 tablet_bootstrap.cc:492] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.241969 8545 tablet_bootstrap.cc:654] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.242198 8552 ts_tablet_manager.cc:1929] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.242257 8552 log.cc:1199] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/d81b3fe6073d4380882057eb97907fbb
I20251028 02:59:55.242462 8552 ts_tablet_manager.cc:1950] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.242568 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 1b436561f0bf44e0b2c680ce30919318 (DEFAULT_TABLE table=default.Table_e3ed44682141426f856f581db3e481af [id=f34cf093333b4460b55dd2a474688449]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.242759 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b436561f0bf44e0b2c680ce30919318. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.242831 8552 tablet_replica.cc:333] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.242933 8552 raft_consensus.cc:2243] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.243033 8552 raft_consensus.cc:2272] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.243219 8546 tablet_bootstrap.cc:492] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:55.243757 8354 leader_election.cc:343] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.244205 8546 tablet_bootstrap.cc:654] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.244776 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "19efc09c08b540a3b42c4107de5acf73" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.245195 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet d81b3fe6073d4380882057eb97907fbb (table default.Table_916c70151f044ff3b34a8e5313675a4f [id=ed2bacfbb3f24d5490ad58320bb33656]) successfully deleted
I20251028 02:59:55.246625 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_48dcb7bf58c748a28999785ae72f248a" } new_table_name: "default.Table_5eddcfdf60ef439397a4948da2d85f90" modify_external_catalogs: true
I20251028 02:59:55.247349 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 12dce4bb3d4b43608d7da9350b14e1d0 with delete_type TABLET_DATA_DELETED (replaced by tablet a808098c70b54ad3a0230352af0b743c) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.248488 8254 tablet_service.cc:1505] Processing CreateTablet for tablet e95a45cb612447dd9bebd9f7a019d1f9 (DEFAULT_TABLE table=default.Table_eb695c5a00aa482d958643fb13e4abaf [id=7e087d7232254c0e87aabe25be6eb25d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.248724 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95a45cb612447dd9bebd9f7a019d1f9. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:55.250743 8356 leader_election.cc:343] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.250850 8356 leader_election.cc:304] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:55.251107 8938 raft_consensus.cc:2749] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:55.251658 8552 ts_tablet_manager.cc:1916] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.252828 9190 raft_consensus.cc:493] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.252908 9190 raft_consensus.cc:515] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.253284 9190 leader_election.cc:290] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.255525 8552 ts_tablet_manager.cc:1929] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.255632 8552 log.cc:1199] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/1f03c8b5dabe49ad8f7fe01c7bbf5d7b
I20251028 02:59:55.255860 8552 ts_tablet_manager.cc:1950] T 1f03c8b5dabe49ad8f7fe01c7bbf5d7b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.256544 8552 tablet_replica.cc:333] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.256763 8552 raft_consensus.cc:2243] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.256791 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 55e72a7f8aa842049da7aeaaa7001a28 (DEFAULT_TABLE table=default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.256933 8552 raft_consensus.cc:2272] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.257105 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55e72a7f8aa842049da7aeaaa7001a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.257499 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9409f3ee379949809f823dfb7f73643b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:55.257738 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d81b3fe6073d4380882057eb97907fbb" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:55.257836 8145 raft_consensus.cc:3060] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.258831 8145 raft_consensus.cc:2468] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:55.259032 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet ec69044a4c3e4f049f90be835cf73367 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.259202 8393 tablet_service.cc:1552] Processing DeleteTablet for tablet 617ca0f545824d14ba7270619f121bcc with delete_type TABLET_DATA_DELETED (replaced by tablet 73c20437695a4043a86faa9c98091e5e) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.259486 8218 leader_election.cc:304] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
W20251028 02:59:55.259589 8218 raft_consensus.cc:2686] T d81b3fe6073d4380882057eb97907fbb P ccbdf478f18948d19de96574a685bfda: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251028 02:59:55.259760 8552 ts_tablet_manager.cc:1916] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:55.259977 8354 leader_election.cc:343] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:55.260066 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9e4eec90a2fa412bb0954805f5b8d832 consensus state has no leader
I20251028 02:59:55.260303 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 4e44345b09584bbca19b19e57ce17c77 from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:55.260347 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06302d2cb5554805b85916741ea10f0c consensus state has no leader
I20251028 02:59:55.260429 8551 tablet_replica.cc:333] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.260569 8551 raft_consensus.cc:2243] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.260668 8551 raft_consensus.cc:2272] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.261003 8255 tablet_service.cc:1552] Processing DeleteTablet for tablet 12dce4bb3d4b43608d7da9350b14e1d0 with delete_type TABLET_DATA_DELETED (replaced by tablet a808098c70b54ad3a0230352af0b743c) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.261390 8551 ts_tablet_manager.cc:1916] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.262677 8261 tablet_service.cc:1505] Processing CreateTablet for tablet f28ca2cf41564792b49940aebc9b2c98 (DEFAULT_TABLE table=default.Table_5008c46b0dcb452497e6dbfd59c37b92 [id=a7962ddee0e841c0ad97786634c03a75]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.262832 8552 ts_tablet_manager.cc:1929] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.262892 8552 log.cc:1199] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/ec69044a4c3e4f049f90be835cf73367
I20251028 02:59:55.262949 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f28ca2cf41564792b49940aebc9b2c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.263108 8552 ts_tablet_manager.cc:1950] T ec69044a4c3e4f049f90be835cf73367 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.263659 8552 tablet_replica.cc:333] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.263794 8552 raft_consensus.cc:2243] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.263833 8551 ts_tablet_manager.cc:1929] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.263901 8551 log.cc:1199] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ec69044a4c3e4f049f90be835cf73367
I20251028 02:59:55.263935 8552 pending_rounds.cc:70] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:55.263969 8552 pending_rounds.cc:77] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597138568343552 op_type: NO_OP noop_request { }
I20251028 02:59:55.264024 8552 raft_consensus.cc:2889] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:55.264071 8552 raft_consensus.cc:2272] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.265275 8393 tablet_service.cc:1552] Processing DeleteTablet for tablet 4246c4bd2abf4569987ca346996445d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 81b521c9f57844bb9873f6b26b5cf661) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.265416 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 4bcdf0368e394d649086042f1fe9774b with delete_type TABLET_DATA_DELETED (replaced by tablet c21f794ec1284f5d99521954ed7ba9c9) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.265419 8551 ts_tablet_manager.cc:1950] T ec69044a4c3e4f049f90be835cf73367 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.265959 8545 tablet_bootstrap.cc:492] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.266047 8545 ts_tablet_manager.cc:1403] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20251028 02:59:55.266392 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b (table default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]) successfully deleted
I20251028 02:59:55.266520 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet ec69044a4c3e4f049f90be835cf73367 (table default.Table_5f8a6ff79e8f4ad7aee8aa99b07f20ba [id=dbed4c8748394b8eb5e82d478f195bf5]) successfully deleted
I20251028 02:59:55.266520 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9409f3ee379949809f823dfb7f73643b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.266651 8545 raft_consensus.cc:359] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.266746 8545 raft_consensus.cc:385] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.266778 8545 raft_consensus.cc:740] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.266893 8545 consensus_queue.cc:260] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.267146 8545 ts_tablet_manager.cc:1434] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.267376 8545 tablet_bootstrap.cc:492] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.267469 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7fa5056d9def461dab21615f8b7e8e82" } modify_external_catalogs: true
I20251028 02:59:55.267568 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7fa5056d9def461dab21615f8b7e8e82" } modify_external_catalogs: true
I20251028 02:59:55.267868 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 55e72a7f8aa842049da7aeaaa7001a28 (DEFAULT_TABLE table=default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.268126 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55e72a7f8aa842049da7aeaaa7001a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.268358 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d611f94943f44256ad3b9ca490f3282c" } new_table_name: "default.Table_594adc002945439ba87c71677fcb4ad1" modify_external_catalogs: true
I20251028 02:59:55.269140 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_78c54cae3ad1472a993dc8ba15399ea2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.269500 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78c54cae3ad1472a993dc8ba15399ea2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.270094 8545 tablet_bootstrap.cc:654] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.270275 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6f0cc2934b86479083615ebdb7ead649"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.270576 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f0cc2934b86479083615ebdb7ead649 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.271360 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_85b63c813dc94719b07f0e12854b164b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.271648 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85b63c813dc94719b07f0e12854b164b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.271947 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_9f6c006d57c74ddd9b5297aa3d310af4" } new_table_name: "default.Table_3af8c6d87c2c4cd1baf98aea5a596a8d" modify_external_catalogs: true
I20251028 02:59:55.272439 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 12dce4bb3d4b43608d7da9350b14e1d0 with delete_type TABLET_DATA_DELETED (replaced by tablet a808098c70b54ad3a0230352af0b743c) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.272756 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 55e72a7f8aa842049da7aeaaa7001a28 (DEFAULT_TABLE table=default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.273212 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 617ca0f545824d14ba7270619f121bcc with delete_type TABLET_DATA_DELETED (replaced by tablet 73c20437695a4043a86faa9c98091e5e) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:55.276222 8356 leader_election.cc:343] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.276319 8356 leader_election.cc:304] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:55.276517 9190 raft_consensus.cc:2749] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:55.277063 8544 tablet_bootstrap.cc:492] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.277156 8544 ts_tablet_manager.cc:1403] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.052s user 0.000s sys 0.002s
I20251028 02:59:55.277645 8544 raft_consensus.cc:359] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.277758 8544 raft_consensus.cc:385] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.277808 8544 raft_consensus.cc:740] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.277956 8544 consensus_queue.cc:260] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.278247 8544 ts_tablet_manager.cc:1434] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:55.278432 8544 tablet_bootstrap.cc:492] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.278750 9382 tablet.cc:1722] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:55.279152 9381 tablet.cc:1722] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:55.281834 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55e72a7f8aa842049da7aeaaa7001a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.282558 8562 raft_consensus.cc:493] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.282650 8562 raft_consensus.cc:515] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.283035 8562 leader_election.cc:290] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.283317 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9409f3ee379949809f823dfb7f73643b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:55.283643 8083 leader_election.cc:343] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.284323 8544 tablet_bootstrap.cc:654] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.284749 8551 tablet_replica.cc:333] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.284894 8551 raft_consensus.cc:2243] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.285005 8551 raft_consensus.cc:2272] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.285498 8551 ts_tablet_manager.cc:1916] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.285636 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9409f3ee379949809f823dfb7f73643b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.285964 8421 raft_consensus.cc:2468] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.286531 8083 leader_election.cc:304] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 02:59:55.286772 9380 tablet.cc:1722] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:55.286996 8562 raft_consensus.cc:2804] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.287062 8562 raft_consensus.cc:493] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.287101 8562 raft_consensus.cc:3060] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.288094 8562 raft_consensus.cc:515] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.288347 8552 ts_tablet_manager.cc:1916] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.288448 8562 leader_election.cc:290] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.288602 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ec69044a4c3e4f049f90be835cf73367 (table default.Table_5f8a6ff79e8f4ad7aee8aa99b07f20ba [id=dbed4c8748394b8eb5e82d478f195bf5]) successfully deleted
I20251028 02:59:55.288748 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9409f3ee379949809f823dfb7f73643b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.288988 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9409f3ee379949809f823dfb7f73643b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.289091 8421 raft_consensus.cc:3060] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.289955 8421 raft_consensus.cc:2468] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.290549 8551 ts_tablet_manager.cc:1929] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.290614 8551 log.cc:1199] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/617ca0f545824d14ba7270619f121bcc
I20251028 02:59:55.290838 8551 ts_tablet_manager.cc:1950] T 617ca0f545824d14ba7270619f121bcc P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.291186 8083 leader_election.cc:304] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.291353 8562 raft_consensus.cc:2804] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.291435 8562 raft_consensus.cc:697] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:55.291565 8562 consensus_queue.cc:237] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.293059 8398 tablet_service.cc:1505] Processing CreateTablet for tablet d39008ac6324451a9d1ed54a2ccc3abf (DEFAULT_TABLE table=default.Table_f3795491c9e547af98943428398101b2 [id=8fe996a5bf644b1a9a0d2d3894445a56]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:55.293923 8083 leader_election.cc:343] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.294188 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 302e0ed455854eb2b661afc2f2b6921d (DEFAULT_TABLE table=default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.294328 8126 tablet_service.cc:1505] Processing CreateTablet for tablet d39008ac6324451a9d1ed54a2ccc3abf (DEFAULT_TABLE table=default.Table_f3795491c9e547af98943428398101b2 [id=8fe996a5bf644b1a9a0d2d3894445a56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.294433 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 302e0ed455854eb2b661afc2f2b6921d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.294692 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d39008ac6324451a9d1ed54a2ccc3abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.295150 8258 tablet_service.cc:1505] Processing CreateTablet for tablet d39008ac6324451a9d1ed54a2ccc3abf (DEFAULT_TABLE table=default.Table_f3795491c9e547af98943428398101b2 [id=8fe996a5bf644b1a9a0d2d3894445a56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.295425 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d39008ac6324451a9d1ed54a2ccc3abf. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:55.295507 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 81c2c40e4da242f8891687a6365d273c consensus state has no leader
I20251028 02:59:55.296386 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 56c8969da9dc464d9b463205e3a27965 (table default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]). Expected version 1 got 0
I20251028 02:59:55.296418 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 302e0ed455854eb2b661afc2f2b6921d (DEFAULT_TABLE table=default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.296644 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 302e0ed455854eb2b661afc2f2b6921d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.298413 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d39008ac6324451a9d1ed54a2ccc3abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.300592 9401 raft_consensus.cc:493] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.300699 9401 raft_consensus.cc:515] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.301092 9401 leader_election.cc:290] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.302017 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 302e0ed455854eb2b661afc2f2b6921d (DEFAULT_TABLE table=default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.302068 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet d9c54e91d2d94573b40b50752318536d with delete_type TABLET_DATA_DELETED (replaced by tablet 35e211a1a8c547baaee1199f7103bfad) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.302287 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 302e0ed455854eb2b661afc2f2b6921d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.305341 8546 tablet_bootstrap.cc:492] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.305446 8546 ts_tablet_manager.cc:1403] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.062s user 0.002s sys 0.000s
I20251028 02:59:55.305935 8546 raft_consensus.cc:359] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.306053 8546 raft_consensus.cc:385] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.306083 8546 raft_consensus.cc:740] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.306191 8546 consensus_queue.cc:260] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.306469 8546 ts_tablet_manager.cc:1434] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.306495 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 102fc67bcf7f4336a26220fdb4f7a4d9
I20251028 02:59:55.306665 8546 tablet_bootstrap.cc:492] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.308158 8546 tablet_bootstrap.cc:654] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.311393 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 102fc67bcf7f4336a26220fdb4f7a4d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.312374 8552 ts_tablet_manager.cc:1929] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.312469 7967 catalog_manager.cc:5649] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:55.313283 8544 tablet_bootstrap.cc:492] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.313397 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 102fc67bcf7f4336a26220fdb4f7a4d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.314330 9401 raft_consensus.cc:493] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.314422 9401 raft_consensus.cc:515] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.314639 8544 ts_tablet_manager.cc:1403] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.001s
I20251028 02:59:55.314752 9401 leader_election.cc:290] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.315197 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fd8682b26c544d58c79f22dd469eedc" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.315376 8421 raft_consensus.cc:2468] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
W20251028 02:59:55.315428 8083 consensus_peers.cc:597] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9860023c7b564342ba3145eafe352beb. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:55.315539 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 102fc67bcf7f4336a26220fdb4f7a4d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:55.315527 8083 leader_election.cc:343] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 617ca0f545824d14ba7270619f121bcc
I20251028 02:59:55.316011 8544 raft_consensus.cc:359] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.316121 8544 raft_consensus.cc:385] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.316155 8544 raft_consensus.cc:740] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.314013 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "617ca0f545824d14ba7270619f121bcc" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.316156 8083 leader_election.cc:304] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.316337 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 617ca0f545824d14ba7270619f121bcc (table default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
I20251028 02:59:55.315256 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fd8682b26c544d58c79f22dd469eedc" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.316527 8283 raft_consensus.cc:2468] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
W20251028 02:59:55.316800 8083 leader_election.cc:343] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: Tablet not RUNNING: STOPPED
I20251028 02:59:55.316871 8083 leader_election.cc:304] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:55.316304 8544 consensus_queue.cc:260] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.317353 8544 ts_tablet_manager.cc:1434] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:55.317528 8544 tablet_bootstrap.cc:492] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.318077 8656 raft_consensus.cc:2749] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:55.316879 8562 raft_consensus.cc:2804] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.318323 8562 raft_consensus.cc:493] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.318377 8562 raft_consensus.cc:3060] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.318466 8544 tablet_bootstrap.cc:654] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.319077 8552 log.cc:1199] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/617ca0f545824d14ba7270619f121bcc
I20251028 02:59:55.319389 8552 ts_tablet_manager.cc:1950] T 617ca0f545824d14ba7270619f121bcc P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.319865 8562 raft_consensus.cc:515] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.320055 8552 tablet_replica.cc:333] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.320184 8552 raft_consensus.cc:2243] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.320161 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 617ca0f545824d14ba7270619f121bcc (table default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
I20251028 02:59:55.320195 8562 leader_election.cc:290] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.320648 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fd8682b26c544d58c79f22dd469eedc" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.320765 8283 raft_consensus.cc:3060] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.321161 8552 raft_consensus.cc:2272] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.321779 8283 raft_consensus.cc:2468] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.321895 8546 tablet_bootstrap.cc:492] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.321967 8546 ts_tablet_manager.cc:1403] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20251028 02:59:55.322103 8083 leader_election.cc:304] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.322415 8546 raft_consensus.cc:359] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.322525 8546 raft_consensus.cc:385] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.322554 8546 raft_consensus.cc:740] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.322655 8546 consensus_queue.cc:260] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.322902 8546 ts_tablet_manager.cc:1434] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:55.323101 8546 tablet_bootstrap.cc:492] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.323091 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fd8682b26c544d58c79f22dd469eedc" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.324004 8552 ts_tablet_manager.cc:1916] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.323215 8417 raft_consensus.cc:3060] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.325213 8417 raft_consensus.cc:2468] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.326063 8562 raft_consensus.cc:2804] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.326162 8562 raft_consensus.cc:697] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:55.326313 8562 consensus_queue.cc:237] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.326989 8546 tablet_bootstrap.cc:654] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:55.328398 8083 consensus_peers.cc:597] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 50e6d41d1cf746aab2802b5c6b2d58b3. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:55.329591 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4e44345b09584bbca19b19e57ce17c77
I20251028 02:59:55.329952 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 4e44345b09584bbca19b19e57ce17c77 deleted and replaced by tablet c8b4b0aa30194ba495adca9138e2454e
I20251028 02:59:55.330150 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1d20b9847b824165a7ad3d5090884817" } modify_external_catalogs: true
I20251028 02:59:55.330262 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1d20b9847b824165a7ad3d5090884817" } modify_external_catalogs: true
I20251028 02:59:55.330817 8546 tablet_bootstrap.cc:492] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.330896 8546 ts_tablet_manager.cc:1403] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
W20251028 02:59:55.331147 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 09d65eef81b2421c92678015b6da4fbb consensus state has no leader
I20251028 02:59:55.331374 8546 raft_consensus.cc:359] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.331471 8546 raft_consensus.cc:385] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.331501 8546 raft_consensus.cc:740] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.331460 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_fabb9e8ebcbc470ca8bc0c3af51bfc25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.316569 8551 tablet_replica.cc:333] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.331615 8546 consensus_queue.cc:260] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.331773 8551 raft_consensus.cc:2243] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.331857 8546 ts_tablet_manager.cc:1434] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 02:59:55.331863 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fabb9e8ebcbc470ca8bc0c3af51bfc25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.331907 8551 raft_consensus.cc:2272] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.332046 8546 tablet_bootstrap.cc:492] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.332458 8551 ts_tablet_manager.cc:1916] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.332778 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.332813 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 4e44345b09584bbca19b19e57ce17c77 with delete_type TABLET_DATA_DELETED (replaced by tablet c8b4b0aa30194ba495adca9138e2454e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.333148 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7972c3d4c7dc428b9efb87313a680606"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.333353 8552 ts_tablet_manager.cc:1929] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.333415 8552 log.cc:1199] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/12dce4bb3d4b43608d7da9350b14e1d0
W20251028 02:59:55.333488 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7972c3d4c7dc428b9efb87313a680606 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.333622 8552 ts_tablet_manager.cc:1950] T 12dce4bb3d4b43608d7da9350b14e1d0 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.334182 8556 raft_consensus.cc:493] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.334260 8556 raft_consensus.cc:515] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.334291 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 12dce4bb3d4b43608d7da9350b14e1d0 (table default.Table_f2b0692abe27469a8f0728978e62ae26 [id=5dd4cb94fa384facbdd1aef677a4b498]) successfully deleted
I20251028 02:59:55.334317 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5affbd47e83d40b2b4628828251725ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.334601 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5affbd47e83d40b2b4628828251725ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.334616 8556 leader_election.cc:290] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.334738 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 4e44345b09584bbca19b19e57ce17c77 with delete_type TABLET_DATA_DELETED (replaced by tablet c8b4b0aa30194ba495adca9138e2454e) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.335196 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9409f3ee379949809f823dfb7f73643b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.335357 8417 raft_consensus.cc:2393] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in this term.
W20251028 02:59:55.335348 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 94cbad596db44060bac2992e9ed17ed8 consensus state has no leader
I20251028 02:59:55.337214 8546 tablet_bootstrap.cc:654] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.337812 8551 ts_tablet_manager.cc:1929] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.337886 8551 log.cc:1199] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4246c4bd2abf4569987ca346996445d4
I20251028 02:59:55.337915 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_26410f48ded64d1285940aeff4f1fc07" } modify_external_catalogs: true
I20251028 02:59:55.338019 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_26410f48ded64d1285940aeff4f1fc07" } modify_external_catalogs: true
I20251028 02:59:55.338120 8551 ts_tablet_manager.cc:1950] T 4246c4bd2abf4569987ca346996445d4 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.338232 8546 tablet_bootstrap.cc:492] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.338304 8546 ts_tablet_manager.cc:1403] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:55.338754 8546 raft_consensus.cc:359] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.338820 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4246c4bd2abf4569987ca346996445d4 (table default.Table_fd3daefb8df048e7bea5949c30da1e1f [id=5f82b9596e5842deb4d45e19c91629bb]) successfully deleted
I20251028 02:59:55.338851 8546 raft_consensus.cc:385] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.338883 8546 raft_consensus.cc:740] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.339006 8546 consensus_queue.cc:260] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.339286 8546 ts_tablet_manager.cc:1434] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.339514 8546 tablet_bootstrap.cc:492] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.340219 8552 tablet_replica.cc:333] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.340322 8552 raft_consensus.cc:2243] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.340428 8552 raft_consensus.cc:2272] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.340433 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c03faadb312742a4aecac0325f3595b7
I20251028 02:59:55.340440 8546 tablet_bootstrap.cc:654] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.341143 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9409f3ee379949809f823dfb7f73643b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:55.341928 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet c03faadb312742a4aecac0325f3595b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.342517 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet c03faadb312742a4aecac0325f3595b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:55.343345 7956 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet a19eeac727e142579e34a301a6e2db02 (table default.Table_0ac514a2be6a4e1b904149fd8fdc8eb3 [id=effb15303ea6421a8eebaa32a124678e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a19eeac727e142579e34a301a6e2db02 consensus state has no leader
I20251028 02:59:55.343767 8218 leader_election.cc:304] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:55.344005 8556 raft_consensus.cc:3060] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.345078 8556 raft_consensus.cc:2749] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:55.345788 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 929cf41ac7b0437387fbc488983f5cac from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.346104 8546 tablet_bootstrap.cc:492] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.346184 8546 ts_tablet_manager.cc:1403] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20251028 02:59:55.346323 8552 ts_tablet_manager.cc:1916] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.346601 8551 tablet_replica.cc:333] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.346788 8551 raft_consensus.cc:2243] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.346968 8551 raft_consensus.cc:2272] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.347272 8546 raft_consensus.cc:359] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.347411 8546 raft_consensus.cc:385] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.347458 8546 raft_consensus.cc:740] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.347627 8546 consensus_queue.cc:260] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.347671 8551 ts_tablet_manager.cc:1916] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.348040 8546 ts_tablet_manager.cc:1434] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:55.349524 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet c03faadb312742a4aecac0325f3595b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.350777 8551 ts_tablet_manager.cc:1929] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.350909 8551 log.cc:1199] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4bcdf0368e394d649086042f1fe9774b
I20251028 02:59:55.351089 8546 tablet_bootstrap.cc:492] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.351466 8551 ts_tablet_manager.cc:1950] T 4bcdf0368e394d649086042f1fe9774b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.351626 7966 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d] alter complete (version 1)
I20251028 02:59:55.352371 8551 tablet_replica.cc:333] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.352573 8551 raft_consensus.cc:2243] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.352814 8551 pending_rounds.cc:70] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
W20251028 02:59:55.352910 8356 consensus_peers.cc:597] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:55.352905 8551 pending_rounds.cc:77] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597138533105664 op_type: NO_OP noop_request { }
I20251028 02:59:55.353194 8551 raft_consensus.cc:2889] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:55.353318 8551 raft_consensus.cc:2272] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.353786 9401 raft_consensus.cc:493] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.353886 9401 raft_consensus.cc:515] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.354256 9401 leader_election.cc:290] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.354910 8546 tablet_bootstrap.cc:654] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.355620 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "429b9cd6b775436a865afd64ec5eae4f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.355736 8283 raft_consensus.cc:2468] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.356099 8083 leader_election.cc:304] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.356168 8546 tablet_bootstrap.cc:492] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.356244 8546 ts_tablet_manager.cc:1403] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:55.356314 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 4e44345b09584bbca19b19e57ce17c77 with delete_type TABLET_DATA_DELETED (replaced by tablet c8b4b0aa30194ba495adca9138e2454e) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.356503 9401 raft_consensus.cc:2804] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.356519 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4bcdf0368e394d649086042f1fe9774b (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]) successfully deleted
I20251028 02:59:55.356562 9401 raft_consensus.cc:493] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.356600 9401 raft_consensus.cc:3060] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.356866 8546 raft_consensus.cc:359] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.356900 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5eddcfdf60ef439397a4948da2d85f90" } modify_external_catalogs: true
I20251028 02:59:55.356969 8546 raft_consensus.cc:385] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.357002 8546 raft_consensus.cc:740] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.357010 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5eddcfdf60ef439397a4948da2d85f90" } modify_external_catalogs: true
I20251028 02:59:55.357138 8546 consensus_queue.cc:260] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.357447 8546 ts_tablet_manager.cc:1434] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.357345 7962 catalog_manager.cc:5649] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:55.357705 9401 raft_consensus.cc:515] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.359022 8544 tablet_bootstrap.cc:492] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.359113 8544 ts_tablet_manager.cc:1403] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.042s user 0.001s sys 0.001s
I20251028 02:59:55.359622 8544 raft_consensus.cc:359] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.359720 8544 raft_consensus.cc:385] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.359751 8544 raft_consensus.cc:740] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.359879 8544 consensus_queue.cc:260] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.359951 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "429b9cd6b775436a865afd64ec5eae4f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.360080 8283 raft_consensus.cc:3060] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.360153 8544 ts_tablet_manager.cc:1434] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.360330 8544 tablet_bootstrap.cc:492] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.361166 8283 raft_consensus.cc:2468] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.361315 8333 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:55.361420 8544 tablet_bootstrap.cc:654] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.361444 9366 raft_consensus.cc:493] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.361522 9366 raft_consensus.cc:515] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.361866 9366 leader_election.cc:290] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:55.365331 8083 leader_election.cc:343] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 4bcdf0368e394d649086042f1fe9774b
I20251028 02:59:55.365767 8546 tablet_bootstrap.cc:492] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.366886 8546 tablet_bootstrap.cc:654] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.367767 8083 leader_election.cc:304] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.368046 8562 raft_consensus.cc:2804] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.368147 8562 raft_consensus.cc:697] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:55.368238 8544 tablet_bootstrap.cc:492] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.368330 8544 ts_tablet_manager.cc:1403] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:55.368290 8562 consensus_queue.cc:237] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.368798 8544 raft_consensus.cc:359] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.368901 8544 raft_consensus.cc:385] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.368937 8544 raft_consensus.cc:740] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.369058 8544 consensus_queue.cc:260] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.369335 8544 ts_tablet_manager.cc:1434] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:55.369505 8544 tablet_bootstrap.cc:492] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.370306 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 68ca4ff164b24b7498f953ec11cf46be
I20251028 02:59:55.370507 8544 tablet_bootstrap.cc:654] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.370991 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 929cf41ac7b0437387fbc488983f5cac
I20251028 02:59:55.371245 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 929cf41ac7b0437387fbc488983f5cac deleted and replaced by tablet b5e3c89222754eacb6ba4773396d62a9
I20251028 02:59:55.371977 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 81c2c40e4da242f8891687a6365d273c
I20251028 02:59:55.372179 8552 ts_tablet_manager.cc:1929] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:55.372273 8552 log.cc:1199] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/102fc67bcf7f4336a26220fdb4f7a4d9
I20251028 02:59:55.372561 8552 ts_tablet_manager.cc:1950] T 102fc67bcf7f4336a26220fdb4f7a4d9 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.372277 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4bcdf0368e394d649086042f1fe9774b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.373324 8283 raft_consensus.cc:2468] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.373603 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 68ca4ff164b24b7498f953ec11cf46be with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.373719 8083 leader_election.cc:304] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:55.374033 8562 raft_consensus.cc:2804] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.374119 8562 raft_consensus.cc:493] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.374161 8562 raft_consensus.cc:3060] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.374192 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 929cf41ac7b0437387fbc488983f5cac with delete_type TABLET_DATA_DELETED (replaced by tablet b5e3c89222754eacb6ba4773396d62a9) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.375080 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 81c2c40e4da242f8891687a6365d273c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.375211 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 102fc67bcf7f4336a26220fdb4f7a4d9 (table default.Table_d6e8f311ccc94210b7a22755a7b4c21a [id=7f26fae434ec4bdda077c8ceb5350aac]) successfully deleted
I20251028 02:59:55.375288 8562 raft_consensus.cc:515] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:55.375406 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4e44345b09584bbca19b19e57ce17c77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4e44345b09584bbca19b19e57ce17c77 already in progress: creating tablet
W20251028 02:59:55.375515 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet c03faadb312742a4aecac0325f3595b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c03faadb312742a4aecac0325f3595b7 already in progress: creating tablet
I20251028 02:59:55.375733 8562 leader_election.cc:290] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.376616 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4bcdf0368e394d649086042f1fe9774b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.376770 8283 raft_consensus.cc:3060] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:55.376767 8083 leader_election.cc:343] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 4bcdf0368e394d649086042f1fe9774b
I20251028 02:59:55.378086 8283 raft_consensus.cc:2468] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.378401 8083 leader_election.cc:304] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:55.378760 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 4bcdf0368e394d649086042f1fe9774b with delete_type TABLET_DATA_DELETED (replaced by tablet c21f794ec1284f5d99521954ed7ba9c9) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.378779 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 68ca4ff164b24b7498f953ec11cf46be with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.378908 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 929cf41ac7b0437387fbc488983f5cac with delete_type TABLET_DATA_DELETED (replaced by tablet b5e3c89222754eacb6ba4773396d62a9) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.378908 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 81c2c40e4da242f8891687a6365d273c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.380159 8550 ts_tablet_manager.cc:1929] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.380234 8550 log.cc:1199] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/7801d57550dc40c39f2d3cf7e11e3713
I20251028 02:59:55.380858 8550 ts_tablet_manager.cc:1950] T 7801d57550dc40c39f2d3cf7e11e3713 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.381704 8550 tablet_replica.cc:333] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.381860 8550 raft_consensus.cc:2243] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.382035 8550 pending_rounds.cc:70] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:55.382069 8550 pending_rounds.cc:77] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597136722882560 op_type: NO_OP noop_request { }
I20251028 02:59:55.382138 8550 raft_consensus.cc:2889] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:55.382189 8550 raft_consensus.cc:2272] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.382629 8552 tablet_replica.cc:333] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.382781 8552 raft_consensus.cc:2243] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.382915 8552 raft_consensus.cc:2272] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.383004 8550 ts_tablet_manager.cc:1916] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.386570 8550 ts_tablet_manager.cc:1929] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.386668 8550 log.cc:1199] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9860023c7b564342ba3145eafe352beb
I20251028 02:59:55.387735 8552 ts_tablet_manager.cc:1916] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.387905 8546 tablet_bootstrap.cc:492] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.387991 8546 ts_tablet_manager.cc:1403] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.002s
I20251028 02:59:55.388501 8546 raft_consensus.cc:359] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.388612 8546 raft_consensus.cc:385] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.388646 8546 raft_consensus.cc:740] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.388788 8546 consensus_queue.cc:260] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.389130 8546 ts_tablet_manager.cc:1434] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.389340 8546 tablet_bootstrap.cc:492] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.390421 8546 tablet_bootstrap.cc:654] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.392314 8546 tablet_bootstrap.cc:492] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.392403 8546 ts_tablet_manager.cc:1403] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20251028 02:59:55.378978 8562 raft_consensus.cc:2804] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.392720 8562 raft_consensus.cc:697] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:55.392867 8562 consensus_queue.cc:237] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.392901 8546 raft_consensus.cc:359] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.392982 8546 raft_consensus.cc:385] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.393011 8546 raft_consensus.cc:740] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.393119 8546 consensus_queue.cc:260] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.393400 8546 ts_tablet_manager.cc:1434] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.393446 8656 raft_consensus.cc:493] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.393530 8656 raft_consensus.cc:515] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.393873 8656 leader_election.cc:290] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.394126 8546 tablet_bootstrap.cc:492] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.395120 8552 ts_tablet_manager.cc:1929] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.395182 8552 log.cc:1199] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/68ca4ff164b24b7498f953ec11cf46be
I20251028 02:59:55.395216 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 7801d57550dc40c39f2d3cf7e11e3713 (table default.Table_416a2058166f4d5a94419c7838d41b75 [id=2d4b8dd08b964bd4a2cc6aec93beff20]) successfully deleted
W20251028 02:59:55.395454 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9a290b8114844aa89b33c75fffc8e1e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a290b8114844aa89b33c75fffc8e1e7 already in progress: creating tablet
I20251028 02:59:55.395536 8546 tablet_bootstrap.cc:654] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:55.395733 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a198db14385543d99afb1881b0fe2409 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a198db14385543d99afb1881b0fe2409 already in progress: creating tablet
I20251028 02:59:55.396184 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9c54e91d2d94573b40b50752318536d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.396310 8284 raft_consensus.cc:2468] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.396657 8083 leader_election.cc:304] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.396854 8656 raft_consensus.cc:2804] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.396914 8656 raft_consensus.cc:493] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.396966 8656 raft_consensus.cc:3060] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.396966 8552 ts_tablet_manager.cc:1950] T 68ca4ff164b24b7498f953ec11cf46be P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.397941 8656 raft_consensus.cc:515] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.398075 8544 tablet_bootstrap.cc:492] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.398150 8544 ts_tablet_manager.cc:1403] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20251028 02:59:55.398320 8656 leader_election.cc:290] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.398607 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 68ca4ff164b24b7498f953ec11cf46be (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:55.398650 8544 raft_consensus.cc:359] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.398738 8544 raft_consensus.cc:385] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251028 02:59:55.398746 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 929cf41ac7b0437387fbc488983f5cac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 929cf41ac7b0437387fbc488983f5cac already in progress: creating tablet
I20251028 02:59:55.398774 8544 raft_consensus.cc:740] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
W20251028 02:59:55.398900 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 81c2c40e4da242f8891687a6365d273c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81c2c40e4da242f8891687a6365d273c already in progress: creating tablet
I20251028 02:59:55.398892 8544 consensus_queue.cc:260] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.399144 8556 raft_consensus.cc:493] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.399214 8544 ts_tablet_manager.cc:1434] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:55.399216 8556 raft_consensus.cc:515] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.399554 8556 leader_election.cc:290] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.400381 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e71f5d12fe2e4822b14c72c2b880c623" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:55.401273 8218 leader_election.cc:343] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.401502 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet ce64e4407ad048e8add422d77de6c211 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.401571 8544 tablet_bootstrap.cc:492] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.401822 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9c54e91d2d94573b40b50752318536d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.401990 8283 raft_consensus.cc:3060] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.402714 8544 tablet_bootstrap.cc:654] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.403672 8283 raft_consensus.cc:2468] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.404407 8083 leader_election.cc:304] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.404673 8656 raft_consensus.cc:2804] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.404757 8656 raft_consensus.cc:697] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:55.404888 8656 consensus_queue.cc:237] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.405627 8556 raft_consensus.cc:493] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.405701 8556 raft_consensus.cc:515] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.406028 8556 leader_election.cc:290] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.406749 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eaa41f8bf7c54673b65893acc59052c1" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:55.407172 8218 leader_election.cc:343] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.410111 8550 ts_tablet_manager.cc:1950] T 9860023c7b564342ba3145eafe352beb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.411005 7966 catalog_manager.cc:5649] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:55.355365 8417 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "429b9cd6b775436a865afd64ec5eae4f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.411620 8417 raft_consensus.cc:2468] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.411645 7964 catalog_manager.cc:5931] Not sending DeleteTablet requests; no consensus state for tablet 51198eb794764f7fbfda806fc256bcca
I20251028 02:59:55.411670 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 7f804f81d70749b5bdefeca4e0bad6fe from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.359424 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "429b9cd6b775436a865afd64ec5eae4f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.411825 8421 raft_consensus.cc:3060] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.412971 8421 raft_consensus.cc:2468] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.374365 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 929cf41ac7b0437387fbc488983f5cac with delete_type TABLET_DATA_DELETED (replaced by tablet b5e3c89222754eacb6ba4773396d62a9) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.374446 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 68ca4ff164b24b7498f953ec11cf46be with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.374737 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 81c2c40e4da242f8891687a6365d273c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.387804 8545 tablet_bootstrap.cc:492] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.394742 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9c54e91d2d94573b40b50752318536d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.413312 8545 ts_tablet_manager.cc:1403] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.146s user 0.002s sys 0.000s
I20251028 02:59:55.413370 8419 raft_consensus.cc:2468] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.399950 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e71f5d12fe2e4822b14c72c2b880c623" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.400954 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d9c54e91d2d94573b40b50752318536d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.413615 8418 raft_consensus.cc:3060] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.413794 8545 raft_consensus.cc:359] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.413885 8545 raft_consensus.cc:385] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.413913 8545 raft_consensus.cc:740] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.414033 8545 consensus_queue.cc:260] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.414283 8545 ts_tablet_manager.cc:1434] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.414476 8545 tablet_bootstrap.cc:492] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.414554 8418 raft_consensus.cc:2468] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.415460 8545 tablet_bootstrap.cc:654] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.415601 8420 raft_consensus.cc:2468] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:55.416312 8220 leader_election.cc:304] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:55.406497 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "eaa41f8bf7c54673b65893acc59052c1" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:55.417644 8220 leader_election.cc:343] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.417716 8220 leader_election.cc:304] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:55.417922 8965 raft_consensus.cc:2804] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.417961 8965 raft_consensus.cc:493] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.417995 8965 raft_consensus.cc:3060] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.418984 8965 raft_consensus.cc:515] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.419400 8965 leader_election.cc:290] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.419495 8965 raft_consensus.cc:2749] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:55.420364 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e71f5d12fe2e4822b14c72c2b880c623" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.420470 8416 raft_consensus.cc:3060] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.420897 8551 ts_tablet_manager.cc:1916] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.421890 8416 raft_consensus.cc:2468] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:55.422875 8469 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:55.423630 8255 tablet_service.cc:1552] Processing DeleteTablet for tablet 4246c4bd2abf4569987ca346996445d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 81b521c9f57844bb9873f6b26b5cf661) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.423763 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet d9c54e91d2d94573b40b50752318536d with delete_type TABLET_DATA_DELETED (replaced by tablet 35e211a1a8c547baaee1199f7103bfad) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.423368 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 4bcdf0368e394d649086042f1fe9774b with delete_type TABLET_DATA_DELETED (replaced by tablet c21f794ec1284f5d99521954ed7ba9c9) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.424054 8552 tablet_replica.cc:333] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.424532 8220 leader_election.cc:304] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:55.424742 8556 raft_consensus.cc:2804] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.424765 8552 raft_consensus.cc:2243] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.425266 8552 raft_consensus.cc:2272] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.424821 8556 raft_consensus.cc:697] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:55.425887 8556 consensus_queue.cc:237] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.427363 8552 ts_tablet_manager.cc:1916] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.411435 9401 leader_election.cc:290] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.425652 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e71f5d12fe2e4822b14c72c2b880c623" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:55.430985 8196 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
W20251028 02:59:55.432303 8218 leader_election.cc:343] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.433030 8550 tablet_replica.cc:333] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.433171 8550 raft_consensus.cc:2243] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.433307 9190 raft_consensus.cc:493] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.433360 8550 raft_consensus.cc:2272] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.433403 9190 raft_consensus.cc:515] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.433805 9190 leader_election.cc:290] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.434450 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ebc0e6d526364d4ba37e443d1e681fbc" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:55.434450 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ebc0e6d526364d4ba37e443d1e681fbc" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.434572 8283 raft_consensus.cc:2468] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:55.434572 8147 raft_consensus.cc:2468] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:55.434882 8356 leader_election.cc:304] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:55.435082 9190 raft_consensus.cc:2804] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.435134 9190 raft_consensus.cc:493] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.435173 9190 raft_consensus.cc:3060] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.436242 9190 raft_consensus.cc:515] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.436609 9190 leader_election.cc:290] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.437487 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet eaa41f8bf7c54673b65893acc59052c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.437145 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ebc0e6d526364d4ba37e443d1e681fbc" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.437626 8283 raft_consensus.cc:3060] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.437597 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ebc0e6d526364d4ba37e443d1e681fbc" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:55.437709 8147 raft_consensus.cc:3060] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.438653 8147 raft_consensus.cc:2468] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:55.438998 8354 leader_election.cc:304] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.439213 8283 raft_consensus.cc:2468] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:55.439189 7967 catalog_manager.cc:5649] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:55.439428 9190 raft_consensus.cc:2804] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.439515 9190 raft_consensus.cc:697] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:55.439670 9190 consensus_queue.cc:237] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:55.440730 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9860023c7b564342ba3145eafe352beb (table default.Table_38f86582ea6e4e5fb73330ab8f1d4c25 [id=b072a25c1a8b4f72ba778ac32c066e71]) successfully deleted
I20251028 02:59:55.441845 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 4125bb9e525c4e7d91ae0862abc44584 (DEFAULT_TABLE table=default.Table_122813ac12be41ba91423e5309fa8eab [id=3131dc70f17c4cdf84d08ab26c718e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.442114 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4125bb9e525c4e7d91ae0862abc44584. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.443043 7964 catalog_manager.cc:5649] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:55.445648 8552 ts_tablet_manager.cc:1929] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.446496 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 7f804f81d70749b5bdefeca4e0bad6fe
I20251028 02:59:55.446897 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 7f804f81d70749b5bdefeca4e0bad6fe deleted and replaced by tablet 5155389485d54c1b82c45eb65364620f
I20251028 02:59:55.447911 8552 log.cc:1199] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4246c4bd2abf4569987ca346996445d4
I20251028 02:59:55.448053 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet d9c54e91d2d94573b40b50752318536d with delete_type TABLET_DATA_DELETED (replaced by tablet 35e211a1a8c547baaee1199f7103bfad) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.448207 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f804f81d70749b5bdefeca4e0bad6fe with delete_type TABLET_DATA_DELETED (replaced by tablet 5155389485d54c1b82c45eb65364620f) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.448609 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f804f81d70749b5bdefeca4e0bad6fe with delete_type TABLET_DATA_DELETED (replaced by tablet 5155389485d54c1b82c45eb65364620f) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.449108 8552 ts_tablet_manager.cc:1950] T 4246c4bd2abf4569987ca346996445d4 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.450418 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 7f804f81d70749b5bdefeca4e0bad6fe with delete_type TABLET_DATA_DELETED (replaced by tablet 5155389485d54c1b82c45eb65364620f) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.452363 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 4246c4bd2abf4569987ca346996445d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 81b521c9f57844bb9873f6b26b5cf661) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.453018 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4246c4bd2abf4569987ca346996445d4 (table default.Table_fd3daefb8df048e7bea5949c30da1e1f [id=5f82b9596e5842deb4d45e19c91629bb]) successfully deleted
I20251028 02:59:55.454337 8552 tablet_replica.cc:333] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.454938 8552 raft_consensus.cc:2243] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:55.455525 8083 consensus_peers.cc:597] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 68ca4ff164b24b7498f953ec11cf46be. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:55.455653 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 4125bb9e525c4e7d91ae0862abc44584 (DEFAULT_TABLE table=default.Table_122813ac12be41ba91423e5309fa8eab [id=3131dc70f17c4cdf84d08ab26c718e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.455631 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 4125bb9e525c4e7d91ae0862abc44584 (DEFAULT_TABLE table=default.Table_122813ac12be41ba91423e5309fa8eab [id=3131dc70f17c4cdf84d08ab26c718e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.455960 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4125bb9e525c4e7d91ae0862abc44584. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.456058 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4125bb9e525c4e7d91ae0862abc44584. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.456092 8550 ts_tablet_manager.cc:1916] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.458487 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 (DEFAULT_TABLE table=default.Table_4f7a6b08bb2b4cee90e54afd8f1b801a [id=243b57c791df41819156363e8607623d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.458760 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e19a2bdc7024b0c9fe84f8983f9d292. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.458729 8552 raft_consensus.cc:2272] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.459968 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ce64e4407ad048e8add422d77de6c211
I20251028 02:59:55.460260 7968 catalog_manager.cc:6719] ReplaceTablet: tablet ce64e4407ad048e8add422d77de6c211 deleted and replaced by tablet a9653fc3236f4e0b812de5a859932587
I20251028 02:59:55.461012 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ef415299e6ef4aa583629acaba69daf0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.461434 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef415299e6ef4aa583629acaba69daf0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.461395 8552 ts_tablet_manager.cc:1916] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.462167 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet ce64e4407ad048e8add422d77de6c211 with delete_type TABLET_DATA_DELETED (replaced by tablet a9653fc3236f4e0b812de5a859932587) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.462939 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet ce64e4407ad048e8add422d77de6c211 with delete_type TABLET_DATA_DELETED (replaced by tablet a9653fc3236f4e0b812de5a859932587) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.465011 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet ce64e4407ad048e8add422d77de6c211 with delete_type TABLET_DATA_DELETED (replaced by tablet a9653fc3236f4e0b812de5a859932587) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.465430 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 (DEFAULT_TABLE table=default.Table_4f7a6b08bb2b4cee90e54afd8f1b801a [id=243b57c791df41819156363e8607623d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.465713 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e19a2bdc7024b0c9fe84f8983f9d292. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.466460 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 (DEFAULT_TABLE table=default.Table_4f7a6b08bb2b4cee90e54afd8f1b801a [id=243b57c791df41819156363e8607623d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.466714 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e19a2bdc7024b0c9fe84f8983f9d292. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.466693 8396 tablet_service.cc:1505] Processing CreateTablet for tablet b9b5fb938357499b8888ba29d23612bd (DEFAULT_TABLE table=default.Table_e1f5688d9de84f4b93cb599116c876bf [id=623eeb8488074e6c96d45caf933c95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.467186 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9b5fb938357499b8888ba29d23612bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.469175 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b14cb211236a485ca1d6b2ef6e476773"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.469416 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_afba81072ed644d4aac9d04bc5902e63" } new_table_name: "default.Table_2547f4c4cdbd4d7f9ec732da77dc545a" modify_external_catalogs: true
W20251028 02:59:55.469588 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b14cb211236a485ca1d6b2ef6e476773 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.469208 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_18a5674a59e34ce4b3bf54efce8a8a44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.471225 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_18a5674a59e34ce4b3bf54efce8a8a44 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.471894 8261 tablet_service.cc:1505] Processing CreateTablet for tablet b9b5fb938357499b8888ba29d23612bd (DEFAULT_TABLE table=default.Table_e1f5688d9de84f4b93cb599116c876bf [id=623eeb8488074e6c96d45caf933c95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.472178 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9b5fb938357499b8888ba29d23612bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.472761 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fabb9e8ebcbc470ca8bc0c3af51bfc25" } new_table_name: "default.Table_8c67370d4f5c47888eda5c234f795044" modify_external_catalogs: true
I20251028 02:59:55.474505 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 464e6b3483bc4816bfa844c52fd3ca53 (DEFAULT_TABLE table=default.Table_096243ad06e04bfba7e8b97d066cdf9c [id=71026b1300014ba6a66c3431c05cc86c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.474843 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 464e6b3483bc4816bfa844c52fd3ca53. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.475174 8544 tablet_bootstrap.cc:492] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.475278 8544 ts_tablet_manager.cc:1403] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.074s user 0.002s sys 0.001s
I20251028 02:59:55.475788 8544 raft_consensus.cc:359] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.475910 8544 raft_consensus.cc:385] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.475955 8544 raft_consensus.cc:740] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.476096 8544 consensus_queue.cc:260] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.476418 8544 ts_tablet_manager.cc:1434] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.476626 8544 tablet_bootstrap.cc:492] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.477667 8544 tablet_bootstrap.cc:654] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:55.480497 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 00550a40f04046d19e9112814372da6c has no consensus state
I20251028 02:59:55.480885 8125 tablet_service.cc:1505] Processing CreateTablet for tablet b9b5fb938357499b8888ba29d23612bd (DEFAULT_TABLE table=default.Table_e1f5688d9de84f4b93cb599116c876bf [id=623eeb8488074e6c96d45caf933c95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.482640 8551 ts_tablet_manager.cc:1929] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.483393 8545 tablet_bootstrap.cc:492] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
W20251028 02:59:55.483482 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13fb883843d141f09aa90e66f9b7bf1f consensus state has no leader
I20251028 02:59:55.483529 8545 ts_tablet_manager.cc:1403] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.069s user 0.002s sys 0.000s
I20251028 02:59:55.483997 8545 raft_consensus.cc:359] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.484109 8545 raft_consensus.cc:385] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.484141 8545 raft_consensus.cc:740] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.484256 8545 consensus_queue.cc:260] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.484553 8545 ts_tablet_manager.cc:1434] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.484795 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9b5fb938357499b8888ba29d23612bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.484853 8545 tablet_bootstrap.cc:492] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.485939 8545 tablet_bootstrap.cc:654] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.490751 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet a198db14385543d99afb1881b0fe2409 with delete_type TABLET_DATA_DELETED (replaced by tablet 302e0ed455854eb2b661afc2f2b6921d) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.491621 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 464e6b3483bc4816bfa844c52fd3ca53 (DEFAULT_TABLE table=default.Table_096243ad06e04bfba7e8b97d066cdf9c [id=71026b1300014ba6a66c3431c05cc86c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.491904 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 464e6b3483bc4816bfa844c52fd3ca53. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.491991 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 464e6b3483bc4816bfa844c52fd3ca53 (DEFAULT_TABLE table=default.Table_096243ad06e04bfba7e8b97d066cdf9c [id=71026b1300014ba6a66c3431c05cc86c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.492149 8550 ts_tablet_manager.cc:1929] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.492211 8550 log.cc:1199] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/50e6d41d1cf746aab2802b5c6b2d58b3
I20251028 02:59:55.492276 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 464e6b3483bc4816bfa844c52fd3ca53. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.493382 8125 tablet_service.cc:1505] Processing CreateTablet for tablet c120a3d56f6b41c28c7f7dcd84359a60 (DEFAULT_TABLE table=default.Table_2b7f5657d1534447a3a29a01698581e1 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.493738 8258 tablet_service.cc:1505] Processing CreateTablet for tablet c120a3d56f6b41c28c7f7dcd84359a60 (DEFAULT_TABLE table=default.Table_2b7f5657d1534447a3a29a01698581e1 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.494102 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c120a3d56f6b41c28c7f7dcd84359a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.494393 8396 tablet_service.cc:1505] Processing CreateTablet for tablet c120a3d56f6b41c28c7f7dcd84359a60 (DEFAULT_TABLE table=default.Table_2b7f5657d1534447a3a29a01698581e1 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.494881 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c120a3d56f6b41c28c7f7dcd84359a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.495743 8551 log.cc:1199] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/12dce4bb3d4b43608d7da9350b14e1d0
I20251028 02:59:55.496091 8551 ts_tablet_manager.cc:1950] T 12dce4bb3d4b43608d7da9350b14e1d0 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.496824 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 12dce4bb3d4b43608d7da9350b14e1d0 (table default.Table_f2b0692abe27469a8f0728978e62ae26 [id=5dd4cb94fa384facbdd1aef677a4b498]) successfully deleted
I20251028 02:59:55.497591 8550 ts_tablet_manager.cc:1950] T 50e6d41d1cf746aab2802b5c6b2d58b3 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.497619 8551 tablet_replica.cc:333] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.497757 8551 raft_consensus.cc:2243] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.497874 8551 raft_consensus.cc:2272] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.498175 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 12eecf12cac641eaad5a9b5af5890941 (DEFAULT_TABLE table=default.Table_47e366fafa11436db14f8076fd1f1ba2 [id=e47909e4e17049549b87f01157c35a4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.498363 8551 ts_tablet_manager.cc:1916] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.498391 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12eecf12cac641eaad5a9b5af5890941. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.499325 8546 tablet_bootstrap.cc:492] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.499418 8546 ts_tablet_manager.cc:1403] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.105s user 0.000s sys 0.002s
I20251028 02:59:55.499660 8545 tablet_bootstrap.cc:492] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.499729 8545 ts_tablet_manager.cc:1403] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 02:59:55.499913 8546 raft_consensus.cc:359] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.500010 8546 raft_consensus.cc:385] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.500043 8546 raft_consensus.cc:740] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.500166 8546 consensus_queue.cc:260] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.500181 8545 raft_consensus.cc:359] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.500269 8545 raft_consensus.cc:385] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.500299 8545 raft_consensus.cc:740] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.500406 8545 consensus_queue.cc:260] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.500464 8546 ts_tablet_manager.cc:1434] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:55.500638 8546 tablet_bootstrap.cc:492] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.500660 8545 ts_tablet_manager.cc:1434] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.500844 8545 tablet_bootstrap.cc:492] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.501513 8546 tablet_bootstrap.cc:654] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.501777 8545 tablet_bootstrap.cc:654] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.506599 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet a198db14385543d99afb1881b0fe2409 with delete_type TABLET_DATA_DELETED (replaced by tablet 302e0ed455854eb2b661afc2f2b6921d) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.506618 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_9bfa5c6f8a5340c1a3df4dfde6854803" } new_table_name: "default.Table_f7fc5927d19b467a91d8c9a63e16d275" modify_external_catalogs: true
I20251028 02:59:55.498733 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c120a3d56f6b41c28c7f7dcd84359a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.507625 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 50e6d41d1cf746aab2802b5c6b2d58b3 (table default.Table_640f9dd799c74ce4bbbc0839fd52b04f [id=2ffdf002bfbe4936b28cd5aefbd24a6a]) successfully deleted
I20251028 02:59:55.508363 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 12eecf12cac641eaad5a9b5af5890941 (DEFAULT_TABLE table=default.Table_47e366fafa11436db14f8076fd1f1ba2 [id=e47909e4e17049549b87f01157c35a4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.508852 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12eecf12cac641eaad5a9b5af5890941. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.509684 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 12eecf12cac641eaad5a9b5af5890941 (DEFAULT_TABLE table=default.Table_47e366fafa11436db14f8076fd1f1ba2 [id=e47909e4e17049549b87f01157c35a4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.509769 8398 tablet_service.cc:1505] Processing CreateTablet for tablet b9803f4cfb9b4ef9a1954c83985d6258 (DEFAULT_TABLE table=default.Table_6c4478bf2de14c3c824d9ab70557b237 [id=82ae3d70b0b544d89bd8e0ff1717f396]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.509974 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12eecf12cac641eaad5a9b5af5890941. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.510026 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9803f4cfb9b4ef9a1954c83985d6258. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.514338 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ea6294c26df241f59b9adcc8f2a9eddc" } modify_external_catalogs: true
I20251028 02:59:55.514478 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ea6294c26df241f59b9adcc8f2a9eddc" } modify_external_catalogs: true
I20251028 02:59:55.515555 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_797c6e55c14f415589c745d9bedfcb07"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.515964 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 3b43da9580244346ae512a7bc9b506d3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b43da9580244346ae512a7bc9b506d3 already in progress: creating tablet
I20251028 02:59:55.515933 8258 tablet_service.cc:1505] Processing CreateTablet for tablet b9803f4cfb9b4ef9a1954c83985d6258 (DEFAULT_TABLE table=default.Table_6c4478bf2de14c3c824d9ab70557b237 [id=82ae3d70b0b544d89bd8e0ff1717f396]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:55.515964 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_797c6e55c14f415589c745d9bedfcb07 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.516536 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9803f4cfb9b4ef9a1954c83985d6258. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:55.517217 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 506396e55fc242ada23d66088984f0ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 506396e55fc242ada23d66088984f0ae already in progress: creating tablet
W20251028 02:59:55.518033 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 83059074c365440b84e39cf7a5553eb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 83059074c365440b84e39cf7a5553eb2 already in progress: creating tablet
W20251028 02:59:55.518530 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 2ab414dbe9144aef8fd72a7f5a95577d because the tablet was not found. No further retry: Not found: Tablet not found: 2ab414dbe9144aef8fd72a7f5a95577d
I20251028 02:59:55.518899 8544 tablet_bootstrap.cc:492] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.519009 8544 ts_tablet_manager.cc:1403] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.000s
W20251028 02:59:55.519187 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1f03c8b5dabe49ad8f7fe01c7bbf5d7b because the tablet was not found. No further retry: Not found: Tablet not found: 1f03c8b5dabe49ad8f7fe01c7bbf5d7b
I20251028 02:59:55.519495 8544 raft_consensus.cc:359] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.519588 8544 raft_consensus.cc:385] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.519613 8544 raft_consensus.cc:740] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.519713 8544 consensus_queue.cc:260] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.519953 8544 ts_tablet_manager.cc:1434] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.519949 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 1d3e422c7ae64b25960714ecc926046a (DEFAULT_TABLE table=default.Table_c7e0f5beff234f17b69504efc678924d [id=181b50c2d29a4b17b8189e3abb10a420]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.520133 8544 tablet_bootstrap.cc:492] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.520195 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3e422c7ae64b25960714ecc926046a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.521255 8544 tablet_bootstrap.cc:654] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.523622 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 1d3e422c7ae64b25960714ecc926046a (DEFAULT_TABLE table=default.Table_c7e0f5beff234f17b69504efc678924d [id=181b50c2d29a4b17b8189e3abb10a420]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.523715 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet a198db14385543d99afb1881b0fe2409 with delete_type TABLET_DATA_DELETED (replaced by tablet 302e0ed455854eb2b661afc2f2b6921d) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.523872 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3e422c7ae64b25960714ecc926046a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.526180 8550 tablet_replica.cc:333] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
W20251028 02:59:55.526202 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 874b6660d86b48ebad03ecc54588da28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 874b6660d86b48ebad03ecc54588da28 already in progress: creating tablet
I20251028 02:59:55.526961 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 1d3e422c7ae64b25960714ecc926046a (DEFAULT_TABLE table=default.Table_c7e0f5beff234f17b69504efc678924d [id=181b50c2d29a4b17b8189e3abb10a420]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.527323 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3e422c7ae64b25960714ecc926046a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.528380 8552 ts_tablet_manager.cc:1929] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.528872 8552 log.cc:1199] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/d9c54e91d2d94573b40b50752318536d
I20251028 02:59:55.529618 8550 raft_consensus.cc:2243] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.529866 8550 pending_rounds.cc:70] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:55.530000 8552 ts_tablet_manager.cc:1950] T d9c54e91d2d94573b40b50752318536d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.530972 8550 pending_rounds.cc:77] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597138598592512 op_type: NO_OP noop_request { }
I20251028 02:59:55.531093 8550 raft_consensus.cc:2889] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:55.531584 8550 raft_consensus.cc:2272] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.532238 8546 tablet_bootstrap.cc:492] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.532325 8546 ts_tablet_manager.cc:1403] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.001s
I20251028 02:59:55.532706 8546 raft_consensus.cc:359] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.532789 8546 raft_consensus.cc:385] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.532827 8546 raft_consensus.cc:740] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.532799 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet d9c54e91d2d94573b40b50752318536d (table default.Table_26410f48ded64d1285940aeff4f1fc07 [id=3191deea26744b009e341282c1babcbf]) successfully deleted
I20251028 02:59:55.532939 8546 consensus_queue.cc:260] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.533198 8546 ts_tablet_manager.cc:1434] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.533484 8546 tablet_bootstrap.cc:492] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.534227 8123 tablet_service.cc:1505] Processing CreateTablet for tablet b9803f4cfb9b4ef9a1954c83985d6258 (DEFAULT_TABLE table=default.Table_6c4478bf2de14c3c824d9ab70557b237 [id=82ae3d70b0b544d89bd8e0ff1717f396]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.534471 8546 tablet_bootstrap.cc:654] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.534499 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9803f4cfb9b4ef9a1954c83985d6258. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.537781 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3ed9eb1b206e4d69ae9e4113517486f9" } new_table_name: "default.Table_f99c9b345302415b87a08fe5c4777310" modify_external_catalogs: true
I20251028 02:59:55.539402 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6e2c063c779e4b639714776f91d8e559"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.539839 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6e2c063c779e4b639714776f91d8e559 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.540025 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7972c3d4c7dc428b9efb87313a680606" } new_table_name: "default.Table_b001b0dab18e4d0a8a4421ddf1f836d9" modify_external_catalogs: true
I20251028 02:59:55.540267 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_215f8629159d41069edc7ce48ed97ae7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.540571 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_215f8629159d41069edc7ce48ed97ae7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.540479 8550 ts_tablet_manager.cc:1916] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.541261 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e067b14c102644fdb1c3a0bbaa79746a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.541522 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e067b14c102644fdb1c3a0bbaa79746a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.542876 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f9254be0618c49f8a6a36fb9fc10354a" } modify_external_catalogs: true
I20251028 02:59:55.542973 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f9254be0618c49f8a6a36fb9fc10354a" } modify_external_catalogs: true
I20251028 02:59:55.544104 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 1ef353f5e715452eae1f1c191fe79d1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9a290b8114844aa89b33c75fffc8e1e7) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.544577 9381 tablet.cc:1722] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:55.544668 8552 tablet_replica.cc:333] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.545190 8552 raft_consensus.cc:2243] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.545555 9380 tablet.cc:1722] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:55.545778 8552 raft_consensus.cc:2272] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.548535 9382 tablet.cc:1722] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:55.548605 8544 tablet_bootstrap.cc:492] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.548673 8544 ts_tablet_manager.cc:1403] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 02:59:55.549180 8544 raft_consensus.cc:359] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.549290 8544 raft_consensus.cc:385] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.549322 8544 raft_consensus.cc:740] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.549448 8544 consensus_queue.cc:260] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.549757 8544 ts_tablet_manager.cc:1434] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:55.549955 8544 tablet_bootstrap.cc:492] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.555706 8552 ts_tablet_manager.cc:1916] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.558480 8552 ts_tablet_manager.cc:1929] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.559887 8550 ts_tablet_manager.cc:1929] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.560020 8550 log.cc:1199] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ec69044a4c3e4f049f90be835cf73367
I20251028 02:59:55.560294 8550 ts_tablet_manager.cc:1950] T ec69044a4c3e4f049f90be835cf73367 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.560745 8544 tablet_bootstrap.cc:654] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.560716 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet ec69044a4c3e4f049f90be835cf73367 (table default.Table_5f8a6ff79e8f4ad7aee8aa99b07f20ba [id=dbed4c8748394b8eb5e82d478f195bf5]) successfully deleted
I20251028 02:59:55.561002 8550 tablet_replica.cc:333] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.561158 8550 raft_consensus.cc:2243] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.561288 8550 raft_consensus.cc:2272] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.561678 8552 log.cc:1199] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4bcdf0368e394d649086042f1fe9774b
I20251028 02:59:55.561853 8550 ts_tablet_manager.cc:1916] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.561918 8552 ts_tablet_manager.cc:1950] T 4bcdf0368e394d649086042f1fe9774b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.562564 8552 tablet_replica.cc:333] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.562659 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4bcdf0368e394d649086042f1fe9774b (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]) successfully deleted
I20251028 02:59:55.562705 8552 raft_consensus.cc:2243] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.562809 8552 raft_consensus.cc:2272] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.563740 8552 ts_tablet_manager.cc:1916] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.563740 8556 raft_consensus.cc:493] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.563866 8556 raft_consensus.cc:515] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.564237 8556 leader_election.cc:290] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.566329 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 67c6ceefa3294af8b68150609f46c117
W20251028 02:59:55.566582 8220 leader_election.cc:343] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: ab1ec08da9f64109a3fe9400fe4e366c
W20251028 02:59:55.566968 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b6d868c81a7e402ba9305bb2aa90625b consensus state has no leader
W20251028 02:59:55.567216 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06f7d43bea27429fb45413e0dc0ba269 has no consensus state
I20251028 02:59:55.567783 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 67c6ceefa3294af8b68150609f46c117 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:55.568431 8218 leader_election.cc:343] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: ab1ec08da9f64109a3fe9400fe4e366c
I20251028 02:59:55.568506 8218 leader_election.cc:304] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:55.568742 8556 raft_consensus.cc:2749] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251028 02:59:55.568977 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 67c6ceefa3294af8b68150609f46c117 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.569294 8552 ts_tablet_manager.cc:1929] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.569378 8552 log.cc:1199] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/eaa41f8bf7c54673b65893acc59052c1
I20251028 02:59:55.569599 8552 ts_tablet_manager.cc:1950] T eaa41f8bf7c54673b65893acc59052c1 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.570019 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 67c6ceefa3294af8b68150609f46c117 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.570382 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet eaa41f8bf7c54673b65893acc59052c1 (table default.Table_33d362b6df3e40b68a3221d89a6b8358 [id=b4c56d84dddf453e96afc2c368def28e]) successfully deleted
I20251028 02:59:55.571905 8550 ts_tablet_manager.cc:1929] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.572003 8550 log.cc:1199] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/d81b3fe6073d4380882057eb97907fbb
I20251028 02:59:55.572224 8550 ts_tablet_manager.cc:1950] T d81b3fe6073d4380882057eb97907fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.572590 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet d81b3fe6073d4380882057eb97907fbb (table default.Table_916c70151f044ff3b34a8e5313675a4f [id=ed2bacfbb3f24d5490ad58320bb33656]) successfully deleted
I20251028 02:59:55.572813 8550 tablet_replica.cc:333] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.572954 8550 raft_consensus.cc:2243] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.573072 8550 raft_consensus.cc:2272] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.573576 8550 ts_tablet_manager.cc:1916] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.574321 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1e1f686b7dda4c03b402b11bae923b8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.570246 8552 tablet_replica.cc:333] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.574698 8552 raft_consensus.cc:2243] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:55.574783 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e1f686b7dda4c03b402b11bae923b8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.574834 8552 raft_consensus.cc:2272] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.575618 8552 ts_tablet_manager.cc:1916] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.574375 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_31e973dfaa9b49f2bed3e9dfd11f7ae9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.576725 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_31e973dfaa9b49f2bed3e9dfd11f7ae9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.577342 8550 ts_tablet_manager.cc:1929] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.577409 8550 log.cc:1199] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/12dce4bb3d4b43608d7da9350b14e1d0
I20251028 02:59:55.577510 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c20f674e0d9a4178a66f16d31bc99638" } modify_external_catalogs: true
I20251028 02:59:55.577621 8550 ts_tablet_manager.cc:1950] T 12dce4bb3d4b43608d7da9350b14e1d0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.577615 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c20f674e0d9a4178a66f16d31bc99638" } modify_external_catalogs: true
I20251028 02:59:55.577839 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_fd8850b2aff9461ca7f69c76704fb00e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.578042 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 12dce4bb3d4b43608d7da9350b14e1d0 (table default.Table_f2b0692abe27469a8f0728978e62ae26 [id=5dd4cb94fa384facbdd1aef677a4b498]) successfully deleted
W20251028 02:59:55.578187 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd8850b2aff9461ca7f69c76704fb00e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.578271 8550 tablet_replica.cc:333] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.578397 8550 raft_consensus.cc:2243] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.578533 8550 raft_consensus.cc:2272] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.578651 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c62970e81d90490ebe817e6ff5c68b97" } modify_external_catalogs: true
I20251028 02:59:55.578744 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c62970e81d90490ebe817e6ff5c68b97" } modify_external_catalogs: true
I20251028 02:59:55.579118 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 55e72a7f8aa842049da7aeaaa7001a28
I20251028 02:59:55.579706 8545 tablet_bootstrap.cc:492] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.579854 8545 ts_tablet_manager.cc:1403] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.079s user 0.002s sys 0.000s
I20251028 02:59:55.580430 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 55e72a7f8aa842049da7aeaaa7001a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.580425 8545 raft_consensus.cc:359] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.581013 8545 raft_consensus.cc:385] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.581097 8545 raft_consensus.cc:740] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.581272 8545 consensus_queue.cc:260] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.581724 8545 ts_tablet_manager.cc:1434] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:55.582049 8545 tablet_bootstrap.cc:492] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.583369 8545 tablet_bootstrap.cc:654] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.584298 8550 ts_tablet_manager.cc:1916] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.584923 8552 ts_tablet_manager.cc:1929] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.585001 8552 log.cc:1199] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/7f804f81d70749b5bdefeca4e0bad6fe
I20251028 02:59:55.585274 8552 ts_tablet_manager.cc:1950] T 7f804f81d70749b5bdefeca4e0bad6fe P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.585256 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet aaaec176a3884a79a55bb74397a7dc9c from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.585987 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_476175b520b24e51a03363f567a515e8 [id=eb0e103661fc4bc7b27bb78fc333fd56]) successfully deleted
I20251028 02:59:55.586047 8552 tablet_replica.cc:333] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.586195 8552 raft_consensus.cc:2243] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.586302 8552 raft_consensus.cc:2272] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:55.586299 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ce64e4407ad048e8add422d77de6c211 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce64e4407ad048e8add422d77de6c211 already in progress: creating tablet
I20251028 02:59:55.588102 8550 ts_tablet_manager.cc:1929] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.588212 8550 log.cc:1199] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/617ca0f545824d14ba7270619f121bcc
I20251028 02:59:55.587960 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 55e72a7f8aa842049da7aeaaa7001a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.588418 8550 ts_tablet_manager.cc:1950] T 617ca0f545824d14ba7270619f121bcc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.588793 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 617ca0f545824d14ba7270619f121bcc (table default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
I20251028 02:59:55.588874 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6dfc5240893546d7a6a5397cd08044dd
I20251028 02:59:55.589751 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 55e72a7f8aa842049da7aeaaa7001a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.590752 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfc5240893546d7a6a5397cd08044dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.591586 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfc5240893546d7a6a5397cd08044dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.591914 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfc5240893546d7a6a5397cd08044dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.592193 8550 tablet_replica.cc:333] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.592321 7966 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_f7fc5927d19b467a91d8c9a63e16d275 [id=7eb5bb61f05d4d79b43385b5e91c5b2e] alter complete (version 2)
I20251028 02:59:55.592358 8550 raft_consensus.cc:2243] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.592499 8550 raft_consensus.cc:2272] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.593389 8550 ts_tablet_manager.cc:1916] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.597406 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 1ef353f5e715452eae1f1c191fe79d1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9a290b8114844aa89b33c75fffc8e1e7) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.598062 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 1ef353f5e715452eae1f1c191fe79d1d with delete_type TABLET_DATA_DELETED (replaced by tablet 9a290b8114844aa89b33c75fffc8e1e7) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.599355 8552 ts_tablet_manager.cc:1916] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.599632 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 1c81698dcbe14d5ba61471e7249705db from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.601338 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet c03faadb312742a4aecac0325f3595b7 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.601773 8552 ts_tablet_manager.cc:1929] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.602247 8550 ts_tablet_manager.cc:1929] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:55.602356 8550 log.cc:1199] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/102fc67bcf7f4336a26220fdb4f7a4d9
I20251028 02:59:55.602613 8550 ts_tablet_manager.cc:1950] T 102fc67bcf7f4336a26220fdb4f7a4d9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.603207 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 102fc67bcf7f4336a26220fdb4f7a4d9 (table default.Table_d6e8f311ccc94210b7a22755a7b4c21a [id=7f26fae434ec4bdda077c8ceb5350aac]) successfully deleted
W20251028 02:59:55.603710 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet c03faadb312742a4aecac0325f3595b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c03faadb312742a4aecac0325f3595b7 already in progress: creating tablet
W20251028 02:59:55.604068 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4e44345b09584bbca19b19e57ce17c77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4e44345b09584bbca19b19e57ce17c77 already in progress: creating tablet
I20251028 02:59:55.604265 8550 tablet_replica.cc:333] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.604357 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f08a6e30211a4d59934c36c7a26ac983
I20251028 02:59:55.604409 8550 raft_consensus.cc:2243] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.604691 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet aaaec176a3884a79a55bb74397a7dc9c
I20251028 02:59:55.605054 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet f08a6e30211a4d59934c36c7a26ac983 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:55.602442 9328 log.cc:927] Time spent T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Append to log took a long time: real 0.171s user 0.000s sys 0.000s
I20251028 02:59:55.605778 8552 log.cc:1199] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/a198db14385543d99afb1881b0fe2409
I20251028 02:59:55.606117 8552 ts_tablet_manager.cc:1950] T a198db14385543d99afb1881b0fe2409 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.606747 8552 tablet_replica.cc:333] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.606895 8552 raft_consensus.cc:2243] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.607024 8552 raft_consensus.cc:2272] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.607151 7967 catalog_manager.cc:6719] ReplaceTablet: tablet aaaec176a3884a79a55bb74397a7dc9c deleted and replaced by tablet 2bf100407b034626b4a9f7e2340f9cf0
I20251028 02:59:55.608209 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet f08a6e30211a4d59934c36c7a26ac983 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.609002 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet a198db14385543d99afb1881b0fe2409 (table default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]) successfully deleted
W20251028 02:59:55.609108 8354 consensus_peers.cc:597] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 -> Peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Couldn't send request to peer fbd2afb4894c4e7a8520cc09f8593aa0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 102fc67bcf7f4336a26220fdb4f7a4d9. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:55.609473 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet aaaec176a3884a79a55bb74397a7dc9c with delete_type TABLET_DATA_DELETED (replaced by tablet 2bf100407b034626b4a9f7e2340f9cf0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.609627 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet aaaec176a3884a79a55bb74397a7dc9c with delete_type TABLET_DATA_DELETED (replaced by tablet 2bf100407b034626b4a9f7e2340f9cf0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.610117 8551 ts_tablet_manager.cc:1929] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.610121 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet aaaec176a3884a79a55bb74397a7dc9c with delete_type TABLET_DATA_DELETED (replaced by tablet 2bf100407b034626b4a9f7e2340f9cf0) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.611063 8551 log.cc:1199] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/d9c54e91d2d94573b40b50752318536d
I20251028 02:59:55.611366 8551 ts_tablet_manager.cc:1950] T d9c54e91d2d94573b40b50752318536d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.612438 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1c81698dcbe14d5ba61471e7249705db
I20251028 02:59:55.612979 8546 tablet_bootstrap.cc:492] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.613090 8546 ts_tablet_manager.cc:1403] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.080s user 0.001s sys 0.001s
I20251028 02:59:55.613126 7964 catalog_manager.cc:6719] ReplaceTablet: tablet 1c81698dcbe14d5ba61471e7249705db deleted and replaced by tablet 40a0431501d348b19ec89fe692741ce5
I20251028 02:59:55.613638 8546 raft_consensus.cc:359] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.613755 8546 raft_consensus.cc:385] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.613770 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c81698dcbe14d5ba61471e7249705db with delete_type TABLET_DATA_DELETED (replaced by tablet 40a0431501d348b19ec89fe692741ce5) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.613797 8546 raft_consensus.cc:740] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.613982 8550 pending_rounds.cc:70] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:55.614020 8550 pending_rounds.cc:77] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597139531042816 op_type: NO_OP noop_request { }
I20251028 02:59:55.613987 8546 consensus_queue.cc:260] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.614084 8550 raft_consensus.cc:2889] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:55.614192 8550 raft_consensus.cc:2272] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.614288 8546 ts_tablet_manager.cc:1434] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.614284 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ce36e2a0a4374085b1e87288b3288ab9" } modify_external_catalogs: true
I20251028 02:59:55.614388 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ce36e2a0a4374085b1e87288b3288ab9" } modify_external_catalogs: true
I20251028 02:59:55.614527 8546 tablet_bootstrap.cc:492] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.614889 8550 ts_tablet_manager.cc:1916] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.615234 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet f08a6e30211a4d59934c36c7a26ac983 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.615599 8546 tablet_bootstrap.cc:654] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.617323 8550 ts_tablet_manager.cc:1929] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.618523 8333 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:55.613924 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c81698dcbe14d5ba61471e7249705db with delete_type TABLET_DATA_DELETED (replaced by tablet 40a0431501d348b19ec89fe692741ce5) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.620280 8551 tablet_replica.cc:333] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.621178 8551 raft_consensus.cc:2243] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.622043 8551 raft_consensus.cc:2272] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.618542 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c81698dcbe14d5ba61471e7249705db with delete_type TABLET_DATA_DELETED (replaced by tablet 40a0431501d348b19ec89fe692741ce5) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.622262 8544 tablet_bootstrap.cc:492] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.622635 8544 ts_tablet_manager.cc:1403] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.073s user 0.003s sys 0.000s
I20251028 02:59:55.623167 8544 raft_consensus.cc:359] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.623276 8544 raft_consensus.cc:385] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.623332 8544 raft_consensus.cc:740] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.623466 8544 consensus_queue.cc:260] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.623654 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 309a9d10178e4984a18e10dfd11cba22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.624019 8544 ts_tablet_manager.cc:1434] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.624035 8551 ts_tablet_manager.cc:1916] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.622561 8550 log.cc:1199] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4bcdf0368e394d649086042f1fe9774b
I20251028 02:59:55.624361 8544 tablet_bootstrap.cc:492] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.625459 8544 tablet_bootstrap.cc:654] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.626173 8546 tablet_bootstrap.cc:492] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.626282 8546 ts_tablet_manager.cc:1403] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:55.626406 8550 ts_tablet_manager.cc:1950] T 4bcdf0368e394d649086042f1fe9774b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.626713 8544 tablet_bootstrap.cc:492] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.626801 8544 ts_tablet_manager.cc:1403] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:55.626803 8546 raft_consensus.cc:359] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.626909 8546 raft_consensus.cc:385] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.626942 8546 raft_consensus.cc:740] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.627154 8546 consensus_queue.cc:260] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.627364 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4bcdf0368e394d649086042f1fe9774b (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]) successfully deleted
I20251028 02:59:55.627489 8544 raft_consensus.cc:359] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.627650 8544 raft_consensus.cc:385] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.627683 8544 raft_consensus.cc:740] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.627866 8550 tablet_replica.cc:333] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.628559 8546 ts_tablet_manager.cc:1434] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:55.628921 8546 tablet_bootstrap.cc:492] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.627889 8544 consensus_queue.cc:260] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.630286 8544 ts_tablet_manager.cc:1434] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:55.630525 8544 tablet_bootstrap.cc:492] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.631129 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 4416e1e10f53401cbddc525d9556d107 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.631625 8544 tablet_bootstrap.cc:654] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.632258 8550 raft_consensus.cc:2243] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.632477 8550 raft_consensus.cc:2272] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.632805 8546 tablet_bootstrap.cc:654] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.632836 8550 ts_tablet_manager.cc:1916] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.634678 8544 tablet_bootstrap.cc:492] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.634874 8544 ts_tablet_manager.cc:1403] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:55.635818 8544 raft_consensus.cc:359] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.636008 8544 raft_consensus.cc:385] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.636184 8544 raft_consensus.cc:740] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.636456 8544 consensus_queue.cc:260] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.637029 8544 ts_tablet_manager.cc:1434] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:55.637405 8544 tablet_bootstrap.cc:492] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.638550 8551 ts_tablet_manager.cc:1929] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:55.638640 8551 log.cc:1199] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/102fc67bcf7f4336a26220fdb4f7a4d9
I20251028 02:59:55.639178 8544 tablet_bootstrap.cc:654] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.640110 8551 ts_tablet_manager.cc:1950] T 102fc67bcf7f4336a26220fdb4f7a4d9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.640391 8545 tablet_bootstrap.cc:492] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.640491 8545 ts_tablet_manager.cc:1403] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.058s user 0.000s sys 0.001s
I20251028 02:59:55.640955 8545 raft_consensus.cc:359] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.641063 8545 raft_consensus.cc:385] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.641108 8545 raft_consensus.cc:740] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.641255 8545 consensus_queue.cc:260] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.641414 8550 ts_tablet_manager.cc:1929] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.641495 8550 log.cc:1199] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/68ca4ff164b24b7498f953ec11cf46be
I20251028 02:59:55.641628 8545 ts_tablet_manager.cc:1434] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251028 02:59:55.641848 8545 tablet_bootstrap.cc:492] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.642333 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 102fc67bcf7f4336a26220fdb4f7a4d9 (table default.Table_d6e8f311ccc94210b7a22755a7b4c21a [id=7f26fae434ec4bdda077c8ceb5350aac]) successfully deleted
I20251028 02:59:55.642757 8550 ts_tablet_manager.cc:1950] T 68ca4ff164b24b7498f953ec11cf46be P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.642884 8545 tablet_bootstrap.cc:654] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.643739 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9bf28d9bd3bd462f864fc4f20468e605
I20251028 02:59:55.645308 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 9bf28d9bd3bd462f864fc4f20468e605 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.645412 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 9bf28d9bd3bd462f864fc4f20468e605 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.645577 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet 9bf28d9bd3bd462f864fc4f20468e605 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.646324 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 68ca4ff164b24b7498f953ec11cf46be (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:55.646750 8546 tablet_bootstrap.cc:492] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
W20251028 02:59:55.646799 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 929cf41ac7b0437387fbc488983f5cac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 929cf41ac7b0437387fbc488983f5cac already in progress: creating tablet
I20251028 02:59:55.646870 8546 ts_tablet_manager.cc:1403] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20251028 02:59:55.646950 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 309a9d10178e4984a18e10dfd11cba22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.647465 8546 raft_consensus.cc:359] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.647583 8546 raft_consensus.cc:385] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.647615 8546 raft_consensus.cc:740] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.647732 8546 consensus_queue.cc:260] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.648015 8546 ts_tablet_manager.cc:1434] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.648223 8546 tablet_bootstrap.cc:492] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:55.649561 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4e44345b09584bbca19b19e57ce17c77 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4e44345b09584bbca19b19e57ce17c77 already in progress: creating tablet
I20251028 02:59:55.649883 8546 tablet_bootstrap.cc:654] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.651738 8544 tablet_bootstrap.cc:492] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.651857 8544 ts_tablet_manager.cc:1403] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 02:59:55.652370 8544 raft_consensus.cc:359] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.652480 8544 raft_consensus.cc:385] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.652513 8544 raft_consensus.cc:740] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.652637 8544 consensus_queue.cc:260] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.652951 8544 ts_tablet_manager.cc:1434] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20251028 02:59:55.653649 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 81c2c40e4da242f8891687a6365d273c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81c2c40e4da242f8891687a6365d273c already in progress: creating tablet
I20251028 02:59:55.654143 8544 tablet_bootstrap.cc:492] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.654656 8550 tablet_replica.cc:333] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.654798 8550 raft_consensus.cc:2243] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.654994 8550 pending_rounds.cc:70] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:55.655071 8550 pending_rounds.cc:77] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597139579703296 op_type: NO_OP noop_request { }
I20251028 02:59:55.655141 8550 raft_consensus.cc:2889] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:55.655200 8550 raft_consensus.cc:2272] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.655753 7963 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet a8f43f58ec624ca4bb12835f2e0eecbf (table default.Table_e2689b73a8454c8dbeab470093ad1463 [id=2f8c8bd6c79d43c894e6356c6e333e7c]). Expected version 1 got 0
I20251028 02:59:55.655889 8550 ts_tablet_manager.cc:1916] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.656852 8123 tablet_service.cc:1552] Processing DeleteTablet for tablet eaa41f8bf7c54673b65893acc59052c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.656935 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 309a9d10178e4984a18e10dfd11cba22 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.658672 7963 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet a8f43f58ec624ca4bb12835f2e0eecbf (table default.Table_e2689b73a8454c8dbeab470093ad1463 [id=2f8c8bd6c79d43c894e6356c6e333e7c]). Expected version 1 got 0
I20251028 02:59:55.660305 8544 tablet_bootstrap.cc:654] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:55.661712 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet c03faadb312742a4aecac0325f3595b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c03faadb312742a4aecac0325f3595b7 already in progress: creating tablet
I20251028 02:59:55.661916 8550 ts_tablet_manager.cc:1929] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.662580 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 9feee73521df42ae89f5af5c5eacf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:55.662631 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 929cf41ac7b0437387fbc488983f5cac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 929cf41ac7b0437387fbc488983f5cac already in progress: creating tablet
I20251028 02:59:55.662824 8551 tablet_replica.cc:333] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.662971 8551 raft_consensus.cc:2243] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.663112 8551 raft_consensus.cc:2272] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.663779 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 49b0b3459d434cf2810029ce205e745c from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.665107 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 8192667f14a44ec296d28efa6057a44d (DEFAULT_TABLE table=default.Table_ef415299e6ef4aa583629acaba69daf0 [id=448356df3cfe4bdcb549b652a04ff06a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.665409 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8192667f14a44ec296d28efa6057a44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.668967 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 8192667f14a44ec296d28efa6057a44d (DEFAULT_TABLE table=default.Table_ef415299e6ef4aa583629acaba69daf0 [id=448356df3cfe4bdcb549b652a04ff06a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.669013 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 593ac7984e01478184af49e5b87ceec4 (DEFAULT_TABLE table=default.Table_b14cb211236a485ca1d6b2ef6e476773 [id=9a874bf3425944c7b30a8a9961f48ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.669209 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 593ac7984e01478184af49e5b87ceec4 (DEFAULT_TABLE table=default.Table_b14cb211236a485ca1d6b2ef6e476773 [id=9a874bf3425944c7b30a8a9961f48ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.669267 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8192667f14a44ec296d28efa6057a44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.669412 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 593ac7984e01478184af49e5b87ceec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.669425 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 593ac7984e01478184af49e5b87ceec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.663535 9401 raft_consensus.cc:493] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.672158 8550 log.cc:1199] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/d9c54e91d2d94573b40b50752318536d
I20251028 02:59:55.672166 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 00550a40f04046d19e9112814372da6c (DEFAULT_TABLE table=default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.672179 9401 raft_consensus.cc:515] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.672302 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 00550a40f04046d19e9112814372da6c (DEFAULT_TABLE table=default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.672402 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00550a40f04046d19e9112814372da6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.672454 8550 ts_tablet_manager.cc:1950] T d9c54e91d2d94573b40b50752318536d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.672523 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00550a40f04046d19e9112814372da6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.672627 9401 leader_election.cc:290] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.665391 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 8192667f14a44ec296d28efa6057a44d (DEFAULT_TABLE table=default.Table_ef415299e6ef4aa583629acaba69daf0 [id=448356df3cfe4bdcb549b652a04ff06a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.672959 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ef353f5e715452eae1f1c191fe79d1d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.673082 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ef353f5e715452eae1f1c191fe79d1d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.673192 8416 raft_consensus.cc:2468] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.673378 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 00550a40f04046d19e9112814372da6c (DEFAULT_TABLE table=default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:55.673484 8083 leader_election.cc:343] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.673767 8083 leader_election.cc:304] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 02:59:55.673928 9401 raft_consensus.cc:2804] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.673990 9401 raft_consensus.cc:493] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.674032 9401 raft_consensus.cc:3060] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.668061 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 593ac7984e01478184af49e5b87ceec4 (DEFAULT_TABLE table=default.Table_b14cb211236a485ca1d6b2ef6e476773 [id=9a874bf3425944c7b30a8a9961f48ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.674988 9401 raft_consensus.cc:515] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.675379 9401 leader_election.cc:290] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.675411 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 2893b3a6e8854187b837a7e6fef58c4e (DEFAULT_TABLE table=default.Table_85b63c813dc94719b07f0e12854b164b [id=49bac53017824831928c69211121863b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.675590 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 2893b3a6e8854187b837a7e6fef58c4e (DEFAULT_TABLE table=default.Table_85b63c813dc94719b07f0e12854b164b [id=49bac53017824831928c69211121863b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.675657 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2893b3a6e8854187b837a7e6fef58c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.675834 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2893b3a6e8854187b837a7e6fef58c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.678212 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ef353f5e715452eae1f1c191fe79d1d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.678309 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1ef353f5e715452eae1f1c191fe79d1d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.678416 8416 raft_consensus.cc:3060] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.678457 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 2893b3a6e8854187b837a7e6fef58c4e (DEFAULT_TABLE table=default.Table_85b63c813dc94719b07f0e12854b164b [id=49bac53017824831928c69211121863b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.679381 8416 raft_consensus.cc:2468] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
W20251028 02:59:55.679921 8083 leader_election.cc:343] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.680295 8393 tablet_service.cc:1505] Processing CreateTablet for tablet ffe824a618714776bfab38679c9565e0 (DEFAULT_TABLE table=default.Table_6f0cc2934b86479083615ebdb7ead649 [id=652e99fe1390470387f6f57273e4e48b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.680449 8121 tablet_service.cc:1505] Processing CreateTablet for tablet ffe824a618714776bfab38679c9565e0 (DEFAULT_TABLE table=default.Table_6f0cc2934b86479083615ebdb7ead649 [id=652e99fe1390470387f6f57273e4e48b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.680461 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 06f7d43bea27429fb45413e0dc0ba269 (DEFAULT_TABLE table=default.Table_b001b0dab18e4d0a8a4421ddf1f836d9 [id=ee6840ec2cd44d6883833d367999752e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.680550 8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe824a618714776bfab38679c9565e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.680685 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06f7d43bea27429fb45413e0dc0ba269. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.680785 8400 tablet_service.cc:1505] Processing CreateTablet for tablet a9653fc3236f4e0b812de5a859932587 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.681025 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9653fc3236f4e0b812de5a859932587. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.682621 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 06f7d43bea27429fb45413e0dc0ba269 (DEFAULT_TABLE table=default.Table_b001b0dab18e4d0a8a4421ddf1f836d9 [id=ee6840ec2cd44d6883833d367999752e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.682865 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06f7d43bea27429fb45413e0dc0ba269. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.683001 8083 leader_election.cc:304] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 02:59:55.683144 8122 tablet_service.cc:1505] Processing CreateTablet for tablet a9653fc3236f4e0b812de5a859932587 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.684444 8258 tablet_service.cc:1505] Processing CreateTablet for tablet a9653fc3236f4e0b812de5a859932587 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.684695 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9653fc3236f4e0b812de5a859932587. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.684887 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 06f7d43bea27429fb45413e0dc0ba269 (DEFAULT_TABLE table=default.Table_b001b0dab18e4d0a8a4421ddf1f836d9 [id=ee6840ec2cd44d6883833d367999752e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.686259 9401 raft_consensus.cc:2804] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.686360 9401 raft_consensus.cc:697] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:55.686506 9401 consensus_queue.cc:237] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.686949 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 66d385494c7a4fb8a6501519925be7d4 (DEFAULT_TABLE table=default.Table_18a5674a59e34ce4b3bf54efce8a8a44 [id=20ee524d16f941159b5c44e85ed885f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.687201 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d385494c7a4fb8a6501519925be7d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.688937 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b50e3ba3bc464d54ba1466ca077c9334"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.689038 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4457c9f159ad428c99a082b56e95f8ad" } new_table_name: "default.Table_6ad31a79d9f043628bdac1dd2d4c263c" modify_external_catalogs: true
W20251028 02:59:55.689404 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b50e3ba3bc464d54ba1466ca077c9334 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.689021 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_326908ab91a545829b75b06816614771"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.690191 9366 raft_consensus.cc:493] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.690270 9366 raft_consensus.cc:515] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.690634 9366 leader_election.cc:290] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:55.690140 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_326908ab91a545829b75b06816614771 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.691659 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_debfc8aa483543dcbc3c9e37d7a16db6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.691795 8083 leader_election.cc:343] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 4246c4bd2abf4569987ca346996445d4
W20251028 02:59:55.691973 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_debfc8aa483543dcbc3c9e37d7a16db6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.691963 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 66d385494c7a4fb8a6501519925be7d4 (DEFAULT_TABLE table=default.Table_18a5674a59e34ce4b3bf54efce8a8a44 [id=20ee524d16f941159b5c44e85ed885f6]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:55.692183 8083 leader_election.cc:343] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: 4246c4bd2abf4569987ca346996445d4
I20251028 02:59:55.692242 8083 leader_election.cc:304] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:55.692377 9366 raft_consensus.cc:2749] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:55.692651 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 66d385494c7a4fb8a6501519925be7d4 (DEFAULT_TABLE table=default.Table_18a5674a59e34ce4b3bf54efce8a8a44 [id=20ee524d16f941159b5c44e85ed885f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.692922 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d385494c7a4fb8a6501519925be7d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.688942 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_086b616472fc449eacfaa348042948e0" } new_table_name: "default.Table_57d84ed216cc40969d5d75d29a6f8f5e" modify_external_catalogs: true
I20251028 02:59:55.693403 8552 ts_tablet_manager.cc:1916] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.695508 8118 tablet_service.cc:1505] Processing CreateTablet for tablet c8b4b0aa30194ba495adca9138e2454e (DEFAULT_TABLE table=default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.696557 8551 ts_tablet_manager.cc:1916] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:55.696769 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aa0776d1a8ef43449309221b86ed3923 consensus state has no leader
I20251028 02:59:55.698035 8258 tablet_service.cc:1505] Processing CreateTablet for tablet c8b4b0aa30194ba495adca9138e2454e (DEFAULT_TABLE table=default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.698287 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8b4b0aa30194ba495adca9138e2454e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.699144 8551 ts_tablet_manager.cc:1929] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.699216 8551 log.cc:1199] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/68ca4ff164b24b7498f953ec11cf46be
I20251028 02:59:55.699548 8551 ts_tablet_manager.cc:1950] T 68ca4ff164b24b7498f953ec11cf46be P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.700261 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 68ca4ff164b24b7498f953ec11cf46be (table default.Table_1d20b9847b824165a7ad3d5090884817 [id=5790180989b54071ad08317318931f3a]) successfully deleted
I20251028 02:59:55.701179 8546 tablet_bootstrap.cc:492] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.701292 8546 ts_tablet_manager.cc:1403] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.000s
I20251028 02:59:55.701437 8396 tablet_service.cc:1505] Processing CreateTablet for tablet aa36c6870c1b4a459f0a41c84855a76e (DEFAULT_TABLE table=default.Table_5affbd47e83d40b2b4628828251725ac [id=3815e9736bd34d668ec7f7e113e11e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.701539 8399 tablet_service.cc:1505] Processing CreateTablet for tablet c8b4b0aa30194ba495adca9138e2454e (DEFAULT_TABLE table=default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.701687 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa36c6870c1b4a459f0a41c84855a76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.701779 8119 tablet_service.cc:1505] Processing CreateTablet for tablet aa36c6870c1b4a459f0a41c84855a76e (DEFAULT_TABLE table=default.Table_5affbd47e83d40b2b4628828251725ac [id=3815e9736bd34d668ec7f7e113e11e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.701870 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8b4b0aa30194ba495adca9138e2454e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.701821 8546 raft_consensus.cc:359] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.701937 8546 raft_consensus.cc:385] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.701973 8546 raft_consensus.cc:740] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.702111 8546 consensus_queue.cc:260] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.702414 8546 ts_tablet_manager.cc:1434] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.702615 8546 tablet_bootstrap.cc:492] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.704222 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 5155389485d54c1b82c45eb65364620f (DEFAULT_TABLE table=default.Table_476175b520b24e51a03363f567a515e8 [id=eb0e103661fc4bc7b27bb78fc333fd56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.704483 8261 tablet_service.cc:1505] Processing CreateTablet for tablet ffe824a618714776bfab38679c9565e0 (DEFAULT_TABLE table=default.Table_6f0cc2934b86479083615ebdb7ead649 [id=652e99fe1390470387f6f57273e4e48b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.704753 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe824a618714776bfab38679c9565e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.705371 8124 tablet_service.cc:1505] Processing CreateTablet for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 (DEFAULT_TABLE table=default.Table_78c54cae3ad1472a993dc8ba15399ea2 [id=9cd25ed7b9a04064b29a42f6fcd95e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.705979 8395 tablet_service.cc:1505] Processing CreateTablet for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 (DEFAULT_TABLE table=default.Table_78c54cae3ad1472a993dc8ba15399ea2 [id=9cd25ed7b9a04064b29a42f6fcd95e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.706255 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.706306 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet a8f43f58ec624ca4bb12835f2e0eecbf (table default.Table_e2689b73a8454c8dbeab470093ad1463 [id=2f8c8bd6c79d43c894e6356c6e333e7c]). Expected version 1 got 0
I20251028 02:59:55.706658 8556 raft_consensus.cc:493] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.706735 8556 raft_consensus.cc:515] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.707086 8556 leader_election.cc:290] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.707755 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4416e1e10f53401cbddc525d9556d107" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:55.708108 8220 leader_election.cc:343] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.709043 8255 tablet_service.cc:1505] Processing CreateTablet for tablet aa36c6870c1b4a459f0a41c84855a76e (DEFAULT_TABLE table=default.Table_5affbd47e83d40b2b4628828251725ac [id=3815e9736bd34d668ec7f7e113e11e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.709183 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4416e1e10f53401cbddc525d9556d107" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:55.709513 8546 tablet_bootstrap.cc:654] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:55.709584 8218 leader_election.cc:343] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.709656 8218 leader_election.cc:304] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:55.709925 8552 ts_tablet_manager.cc:1929] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.709987 8552 log.cc:1199] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/1ef353f5e715452eae1f1c191fe79d1d
I20251028 02:59:55.710109 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 5bf9795bd59b42ea997a97c84e1ea111 (DEFAULT_TABLE table=default.Table_7bfb43f76f8b46b292deee50a8f58280 [id=473a7ccabea349c087076b5d741690f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.710198 8552 ts_tablet_manager.cc:1950] T 1ef353f5e715452eae1f1c191fe79d1d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.710336 8556 raft_consensus.cc:2749] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:55.710536 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 5155389485d54c1b82c45eb65364620f (DEFAULT_TABLE table=default.Table_476175b520b24e51a03363f567a515e8 [id=eb0e103661fc4bc7b27bb78fc333fd56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.710768 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5155389485d54c1b82c45eb65364620f. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:55.711091 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9e4eec90a2fa412bb0954805f5b8d832 consensus state has no leader
I20251028 02:59:55.711618 8257 tablet_service.cc:1505] Processing CreateTablet for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 (DEFAULT_TABLE table=default.Table_78c54cae3ad1472a993dc8ba15399ea2 [id=9cd25ed7b9a04064b29a42f6fcd95e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.711834 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.712033 8545 tablet_bootstrap.cc:492] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.712119 8545 ts_tablet_manager.cc:1403] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.070s user 0.002s sys 0.000s
I20251028 02:59:55.712316 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 5bf9795bd59b42ea997a97c84e1ea111 (DEFAULT_TABLE table=default.Table_7bfb43f76f8b46b292deee50a8f58280 [id=473a7ccabea349c087076b5d741690f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.712558 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bf9795bd59b42ea997a97c84e1ea111. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.712594 8545 raft_consensus.cc:359] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.712639 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet a8f43f58ec624ca4bb12835f2e0eecbf (table default.Table_e2689b73a8454c8dbeab470093ad1463 [id=2f8c8bd6c79d43c894e6356c6e333e7c]). Expected version 1 got 0
I20251028 02:59:55.712705 8545 raft_consensus.cc:385] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.712738 8545 raft_consensus.cc:740] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.712899 8545 consensus_queue.cc:260] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.713202 8545 ts_tablet_manager.cc:1434] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.713413 8545 tablet_bootstrap.cc:492] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.716387 8545 tablet_bootstrap.cc:654] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.717114 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet eaa41f8bf7c54673b65893acc59052c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.717293 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 5bf9795bd59b42ea997a97c84e1ea111 (DEFAULT_TABLE table=default.Table_7bfb43f76f8b46b292deee50a8f58280 [id=473a7ccabea349c087076b5d741690f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.717547 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bf9795bd59b42ea997a97c84e1ea111. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.717643 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 5155389485d54c1b82c45eb65364620f (DEFAULT_TABLE table=default.Table_476175b520b24e51a03363f567a515e8 [id=eb0e103661fc4bc7b27bb78fc333fd56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.717902 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5155389485d54c1b82c45eb65364620f. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:55.720116 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 81c2c40e4da242f8891687a6365d273c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81c2c40e4da242f8891687a6365d273c already in progress: creating tablet
I20251028 02:59:55.720350 8551 tablet_replica.cc:333] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.720489 8551 raft_consensus.cc:2243] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.720695 8551 raft_consensus.cc:2272] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.722065 8551 ts_tablet_manager.cc:1916] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.722174 8115 tablet_service.cc:1505] Processing CreateTablet for tablet b5e3c89222754eacb6ba4773396d62a9 (DEFAULT_TABLE table=default.Table_aa25a85ac8d24940a727a586d6c630db [id=10f40fa1652f495e91fcbd5852bc3464]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.722723 8258 tablet_service.cc:1505] Processing CreateTablet for tablet b5e3c89222754eacb6ba4773396d62a9 (DEFAULT_TABLE table=default.Table_aa25a85ac8d24940a727a586d6c630db [id=10f40fa1652f495e91fcbd5852bc3464]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.723023 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e3c89222754eacb6ba4773396d62a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.723886 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 1ef353f5e715452eae1f1c191fe79d1d (table default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]) successfully deleted
I20251028 02:59:55.724649 8552 tablet_replica.cc:333] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.724679 8395 tablet_service.cc:1505] Processing CreateTablet for tablet b5e3c89222754eacb6ba4773396d62a9 (DEFAULT_TABLE table=default.Table_aa25a85ac8d24940a727a586d6c630db [id=10f40fa1652f495e91fcbd5852bc3464]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.724783 8552 raft_consensus.cc:2243] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.724929 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e3c89222754eacb6ba4773396d62a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.725476 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa36c6870c1b4a459f0a41c84855a76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.725622 8552 raft_consensus.cc:2272] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.726660 8546 tablet_bootstrap.cc:492] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.726747 8546 ts_tablet_manager.cc:1403] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20251028 02:59:55.727242 8546 raft_consensus.cc:359] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.727380 8546 raft_consensus.cc:385] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.727421 8546 raft_consensus.cc:740] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.727560 8546 consensus_queue.cc:260] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.727855 8546 ts_tablet_manager.cc:1434] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.728067 8546 tablet_bootstrap.cc:492] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.729162 8546 tablet_bootstrap.cc:654] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.729321 8551 ts_tablet_manager.cc:1929] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.729390 8551 log.cc:1199] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/7f804f81d70749b5bdefeca4e0bad6fe
I20251028 02:59:55.729640 8551 ts_tablet_manager.cc:1950] T 7f804f81d70749b5bdefeca4e0bad6fe P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.730324 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_476175b520b24e51a03363f567a515e8 [id=eb0e103661fc4bc7b27bb78fc333fd56]) successfully deleted
I20251028 02:59:55.731945 8552 ts_tablet_manager.cc:1916] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:55.732381 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ce64e4407ad048e8add422d77de6c211 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce64e4407ad048e8add422d77de6c211 already in progress: creating tablet
I20251028 02:59:55.732610 8551 tablet_replica.cc:333] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.732774 8551 raft_consensus.cc:2243] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.732911 8551 raft_consensus.cc:2272] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.732970 9366 raft_consensus.cc:493] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.733075 9366 raft_consensus.cc:515] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.733440 9366 leader_election.cc:290] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.733498 8551 ts_tablet_manager.cc:1916] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.733920 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "19efc09c08b540a3b42c4107de5acf73" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.734041 8283 raft_consensus.cc:2468] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.734042 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "19efc09c08b540a3b42c4107de5acf73" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.734164 8416 raft_consensus.cc:2468] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.734341 8083 leader_election.cc:304] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.734763 9366 raft_consensus.cc:2804] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.734830 9366 raft_consensus.cc:493] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.734866 9366 raft_consensus.cc:3060] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.735751 8551 ts_tablet_manager.cc:1929] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.735819 8551 log.cc:1199] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a198db14385543d99afb1881b0fe2409
I20251028 02:59:55.735885 9366 raft_consensus.cc:515] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.736049 8551 ts_tablet_manager.cc:1950] T a198db14385543d99afb1881b0fe2409 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.736239 9366 leader_election.cc:290] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.736712 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a198db14385543d99afb1881b0fe2409 (table default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]) successfully deleted
I20251028 02:59:55.736860 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "19efc09c08b540a3b42c4107de5acf73" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.736937 8551 tablet_replica.cc:333] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.736979 8283 raft_consensus.cc:3060] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.737171 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "19efc09c08b540a3b42c4107de5acf73" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.737241 8551 raft_consensus.cc:2243] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.737283 8416 raft_consensus.cc:3060] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.737353 8551 raft_consensus.cc:2272] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.737855 8551 ts_tablet_manager.cc:1916] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.737915 8283 raft_consensus.cc:2468] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.738212 8083 leader_election.cc:304] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.738236 8416 raft_consensus.cc:2468] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.738572 9366 raft_consensus.cc:2804] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.738654 9366 raft_consensus.cc:697] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:55.738780 9366 consensus_queue.cc:237] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.739411 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet ce64e4407ad048e8add422d77de6c211 with delete_type TABLET_DATA_DELETED (replaced by tablet a9653fc3236f4e0b812de5a859932587) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.739830 8551 ts_tablet_manager.cc:1929] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.739894 8551 log.cc:1199] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/67c6ceefa3294af8b68150609f46c117
I20251028 02:59:55.740448 8551 ts_tablet_manager.cc:1950] T 67c6ceefa3294af8b68150609f46c117 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.741040 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00550a40f04046d19e9112814372da6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.741184 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2893b3a6e8854187b837a7e6fef58c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.741421 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bf9795bd59b42ea997a97c84e1ea111. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.741580 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d385494c7a4fb8a6501519925be7d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.741707 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e3c89222754eacb6ba4773396d62a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.741770 8550 tablet_replica.cc:333] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.741815 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9653fc3236f4e0b812de5a859932587. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.742029 8550 raft_consensus.cc:2243] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.742046 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8b4b0aa30194ba495adca9138e2454e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.742317 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe824a618714776bfab38679c9565e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.742458 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.743646 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet d9c54e91d2d94573b40b50752318536d (table default.Table_26410f48ded64d1285940aeff4f1fc07 [id=3191deea26744b009e341282c1babcbf]) successfully deleted
I20251028 02:59:55.744441 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 593ac7984e01478184af49e5b87ceec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.744700 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06f7d43bea27429fb45413e0dc0ba269. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.746374 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5155389485d54c1b82c45eb65364620f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.746546 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa36c6870c1b4a459f0a41c84855a76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.746891 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e474b4516d5444bbb4b48bcbbbbf7cc1" } new_table_name: "default.Table_cdab567cf8da46e8a4d7ba2a80e1e2d9" modify_external_catalogs: true
I20251028 02:59:55.747665 8550 raft_consensus.cc:2272] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.748869 8544 tablet_bootstrap.cc:492] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.748942 8544 ts_tablet_manager.cc:1403] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.095s user 0.002s sys 0.000s
I20251028 02:59:55.749076 8550 ts_tablet_manager.cc:1916] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.749430 8544 raft_consensus.cc:359] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.749524 8544 raft_consensus.cc:385] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.749554 8544 raft_consensus.cc:740] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.749667 8544 consensus_queue.cc:260] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.749954 8544 ts_tablet_manager.cc:1434] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.750108 8544 tablet_bootstrap.cc:492] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.751072 8544 tablet_bootstrap.cc:654] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.751437 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8192667f14a44ec296d28efa6057a44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.752928 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 67c6ceefa3294af8b68150609f46c117 (table default.Table_f9254be0618c49f8a6a36fb9fc10354a [id=86c2f59df3dd4d2195d18835dceff79b]) successfully deleted
W20251028 02:59:55.753441 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 55e72a7f8aa842049da7aeaaa7001a28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55e72a7f8aa842049da7aeaaa7001a28 already in progress: creating tablet
W20251028 02:59:55.753831 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 6dfc5240893546d7a6a5397cd08044dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6dfc5240893546d7a6a5397cd08044dd already in progress: creating tablet
I20251028 02:59:55.754045 8551 tablet_replica.cc:333] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
W20251028 02:59:55.754272 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1979906599754f7ba049c7a9b23bf26d consensus state has no leader
I20251028 02:59:55.754297 8551 raft_consensus.cc:2243] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.754758 8551 raft_consensus.cc:2272] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.755394 8551 ts_tablet_manager.cc:1916] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.756127 8546 tablet_bootstrap.cc:492] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.756229 8546 ts_tablet_manager.cc:1403] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20251028 02:59:55.756449 8656 consensus_queue.cc:1048] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:55.756796 8546 raft_consensus.cc:359] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.756913 8546 raft_consensus.cc:385] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.756955 8546 raft_consensus.cc:740] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.757086 8546 consensus_queue.cc:260] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.757354 8546 ts_tablet_manager.cc:1434] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.757578 8546 tablet_bootstrap.cc:492] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.757867 8551 ts_tablet_manager.cc:1929] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.757931 8551 log.cc:1199] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/1ef353f5e715452eae1f1c191fe79d1d
I20251028 02:59:55.758145 8551 ts_tablet_manager.cc:1950] T 1ef353f5e715452eae1f1c191fe79d1d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.758970 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 1ef353f5e715452eae1f1c191fe79d1d (table default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]) successfully deleted
W20251028 02:59:55.759433 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f08a6e30211a4d59934c36c7a26ac983 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f08a6e30211a4d59934c36c7a26ac983 already in progress: creating tablet
W20251028 02:59:55.759804 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet aaaec176a3884a79a55bb74397a7dc9c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aaaec176a3884a79a55bb74397a7dc9c already in progress: creating tablet
I20251028 02:59:55.760000 8551 tablet_replica.cc:333] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.760140 8551 raft_consensus.cc:2243] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.760275 8551 raft_consensus.cc:2272] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.760701 8551 ts_tablet_manager.cc:1916] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.762539 9401 consensus_queue.cc:1048] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:55.761704 8255 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a81c8bc182442bd8596251b99f90a2d with delete_type TABLET_DATA_DELETED (replaced by tablet c1470187cdc74716b727052ea2b0b4a0) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.763147 8545 tablet_bootstrap.cc:492] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.763242 8545 ts_tablet_manager.cc:1403] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.050s user 0.001s sys 0.000s
I20251028 02:59:55.763767 8545 raft_consensus.cc:359] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.763862 8545 raft_consensus.cc:385] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.763899 8545 raft_consensus.cc:740] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.764034 8545 consensus_queue.cc:260] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.764343 8545 ts_tablet_manager.cc:1434] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.764534 8545 tablet_bootstrap.cc:492] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.764937 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f4e5df9294de4390bfbaaf9e86ade7c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.765388 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4e5df9294de4390bfbaaf9e86ade7c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.765563 8545 tablet_bootstrap.cc:654] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.766259 8546 tablet_bootstrap.cc:654] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.767936 8551 ts_tablet_manager.cc:1929] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20251028 02:59:55.768011 8553 raft_consensus.cc:493] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.768136 8553 raft_consensus.cc:515] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.768478 8553 leader_election.cc:290] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.768554 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_8314d35830564f179d7e69e756fefa45"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.768879 8551 log.cc:1199] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/1c81698dcbe14d5ba61471e7249705db
W20251028 02:59:55.768970 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8314d35830564f179d7e69e756fefa45 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.769338 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_12b6b08740be470382334a74e5df73cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.769560 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a8f43f58ec624ca4bb12835f2e0eecbf" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:55.769654 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12b6b08740be470382334a74e5df73cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.769665 8420 raft_consensus.cc:2468] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:55.769840 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_50c26dd07fa04d9a838968de2a92f0e5" } new_table_name: "default.Table_5002cbe7bf3044e4baaa8e884eb9283f" modify_external_catalogs: true
I20251028 02:59:55.769963 8220 leader_election.cc:304] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:55.770115 8556 raft_consensus.cc:2804] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.770153 8556 raft_consensus.cc:493] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.770187 8556 raft_consensus.cc:3060] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.770961 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_476175b520b24e51a03363f567a515e8" } new_table_name: "default.Table_b359d8a7e6724c21a2f0de83ddd841b7" modify_external_catalogs: true
I20251028 02:59:55.771189 8556 raft_consensus.cc:515] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.771574 8556 leader_election.cc:290] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.771977 8551 ts_tablet_manager.cc:1950] T 1c81698dcbe14d5ba61471e7249705db P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.772099 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a8f43f58ec624ca4bb12835f2e0eecbf" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.772223 8420 raft_consensus.cc:3060] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.772485 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4b9a111a46cd448cac4abf954888e213" } modify_external_catalogs: true
I20251028 02:59:55.772584 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4b9a111a46cd448cac4abf954888e213" } modify_external_catalogs: true
I20251028 02:59:55.773180 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a8f43f58ec624ca4bb12835f2e0eecbf" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:55.773247 8420 raft_consensus.cc:2468] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:55.773260 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a8f43f58ec624ca4bb12835f2e0eecbf" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:55.773319 8147 raft_consensus.cc:2468] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:55.773538 8220 leader_election.cc:304] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:55.773947 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 1c81698dcbe14d5ba61471e7249705db (table default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52]) successfully deleted
I20251028 02:59:55.774394 8469 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
W20251028 02:59:55.775223 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 9bf28d9bd3bd462f864fc4f20468e605 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9bf28d9bd3bd462f864fc4f20468e605 already in progress: creating tablet
I20251028 02:59:55.776185 8556 raft_consensus.cc:2804] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.776274 8556 raft_consensus.cc:697] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:55.776397 8556 consensus_queue.cc:237] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.776983 8551 tablet_replica.cc:333] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.777010 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4045ef6d27fd4f2fbfb8975411994305"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.777136 8551 raft_consensus.cc:2243] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.777226 8551 raft_consensus.cc:2272] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:55.777400 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4045ef6d27fd4f2fbfb8975411994305 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.777777 8551 ts_tablet_manager.cc:1916] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.778564 8562 raft_consensus.cc:493] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.778659 8562 raft_consensus.cc:515] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.779036 8562 leader_election.cc:290] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:55.780735 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 74120decd53d4de19c5f6fedf70ecaae consensus state has no leader
W20251028 02:59:55.780740 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5155389485d54c1b82c45eb65364620f consensus state has no leader
I20251028 02:59:55.781561 8656 consensus_queue.cc:1048] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:55.781972 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3292bf8a27264e8491009af851138333" } modify_external_catalogs: true
I20251028 02:59:55.782088 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3292bf8a27264e8491009af851138333" } modify_external_catalogs: true
I20251028 02:59:55.782071 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e71f5d12fe2e4822b14c72c2b880c623" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.782217 8420 raft_consensus.cc:2393] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:55.780966 8550 ts_tablet_manager.cc:1929] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.782393 8550 log.cc:1199] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/7f804f81d70749b5bdefeca4e0bad6fe
I20251028 02:59:55.782364 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e71f5d12fe2e4822b14c72c2b880c623" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.782797 8550 ts_tablet_manager.cc:1950] T 7f804f81d70749b5bdefeca4e0bad6fe P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.785207 8083 leader_election.cc:304] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:55.786900 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e9eca9a92e57463bbf1e10276cd7e50a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.787241 8562 raft_consensus.cc:3060] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:55.787364 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9eca9a92e57463bbf1e10276cd7e50a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.787737 8550 tablet_replica.cc:333] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.787865 8550 raft_consensus.cc:2243] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.787964 8550 raft_consensus.cc:2272] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.788071 8551 ts_tablet_manager.cc:1929] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.788154 8551 log.cc:1199] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/309a9d10178e4984a18e10dfd11cba22
I20251028 02:59:55.788262 8562 raft_consensus.cc:2749] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:55.788388 8551 ts_tablet_manager.cc:1950] T 309a9d10178e4984a18e10dfd11cba22 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.789160 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 309a9d10178e4984a18e10dfd11cba22 (table default.Table_c6ab20b423864891ba7a399af893a205 [id=4e185be67d91416e835dfc1c32e7e1e8]) successfully deleted
I20251028 02:59:55.789204 8545 tablet_bootstrap.cc:492] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.789291 8545 ts_tablet_manager.cc:1403] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20251028 02:59:55.789680 8551 tablet_replica.cc:333] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.789830 8545 raft_consensus.cc:359] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.789863 8656 consensus_queue.cc:1048] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:55.789940 8545 raft_consensus.cc:385] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.789971 8545 raft_consensus.cc:740] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.790082 8545 consensus_queue.cc:260] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.790373 8545 ts_tablet_manager.cc:1434] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:55.790552 8545 tablet_bootstrap.cc:492] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.791819 8545 tablet_bootstrap.cc:654] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.789835 8551 raft_consensus.cc:2243] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.792524 8551 raft_consensus.cc:2272] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.792593 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 7f804f81d70749b5bdefeca4e0bad6fe (table default.Table_b359d8a7e6724c21a2f0de83ddd841b7 [id=eb0e103661fc4bc7b27bb78fc333fd56]) successfully deleted
I20251028 02:59:55.792622 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_aeee4c7fa7ae41beb0ef15ad542213e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.792690 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e2689b73a8454c8dbeab470093ad1463" } modify_external_catalogs: true
I20251028 02:59:55.792812 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e2689b73a8454c8dbeab470093ad1463" } modify_external_catalogs: true
W20251028 02:59:55.793061 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aeee4c7fa7ae41beb0ef15ad542213e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.793113 8550 ts_tablet_manager.cc:1916] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.795403 8550 ts_tablet_manager.cc:1929] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.796481 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet df60914093f444fda88c13a132587513 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.798539 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1562d90171b44a268e4777c51b3c8483"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.798954 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1562d90171b44a268e4777c51b3c8483 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.808228 8551 ts_tablet_manager.cc:1916] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.809919 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b359d8a7e6724c21a2f0de83ddd841b7" } modify_external_catalogs: true
I20251028 02:59:55.810073 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b359d8a7e6724c21a2f0de83ddd841b7" } modify_external_catalogs: true
I20251028 02:59:55.810743 8551 ts_tablet_manager.cc:1929] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.810829 8551 log.cc:1199] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/eaa41f8bf7c54673b65893acc59052c1
I20251028 02:59:55.795480 8550 log.cc:1199] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4246c4bd2abf4569987ca346996445d4
I20251028 02:59:55.811589 8550 ts_tablet_manager.cc:1950] T 4246c4bd2abf4569987ca346996445d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.812145 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4246c4bd2abf4569987ca346996445d4 (table default.Table_fd3daefb8df048e7bea5949c30da1e1f [id=5f82b9596e5842deb4d45e19c91629bb]) successfully deleted
I20251028 02:59:55.812438 8550 tablet_replica.cc:333] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
W20251028 02:59:55.812434 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ce64e4407ad048e8add422d77de6c211 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce64e4407ad048e8add422d77de6c211 already in progress: creating tablet
I20251028 02:59:55.812597 8550 raft_consensus.cc:2243] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.812729 8550 raft_consensus.cc:2272] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.813242 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet ebc0e6d526364d4ba37e443d1e681fbc from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.813496 8544 tablet_bootstrap.cc:492] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.813589 8550 ts_tablet_manager.cc:1916] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.813617 8544 ts_tablet_manager.cc:1403] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.064s user 0.002s sys 0.000s
I20251028 02:59:55.815768 8544 raft_consensus.cc:359] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.817274 8544 raft_consensus.cc:385] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.817397 8544 raft_consensus.cc:740] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.817626 8544 consensus_queue.cc:260] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.818069 8544 ts_tablet_manager.cc:1434] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:55.818944 8544 tablet_bootstrap.cc:492] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.820813 8550 ts_tablet_manager.cc:1929] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.820932 8544 tablet_bootstrap.cc:654] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.820971 8550 log.cc:1199] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a198db14385543d99afb1881b0fe2409
I20251028 02:59:55.821331 8562 raft_consensus.cc:493] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.821432 8562 raft_consensus.cc:515] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.822263 8562 leader_election.cc:290] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.822866 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9feee73521df42ae89f5af5c5eacf9bd" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.823011 8284 raft_consensus.cc:2468] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.823355 8083 leader_election.cc:304] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.822746 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9feee73521df42ae89f5af5c5eacf9bd" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.824298 8562 raft_consensus.cc:2804] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.824599 8562 raft_consensus.cc:493] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.824657 8562 raft_consensus.cc:3060] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.825637 8562 raft_consensus.cc:515] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.825708 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet b993932fd567416984ad6d28f8cadad6 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.825896 8551 ts_tablet_manager.cc:1950] T eaa41f8bf7c54673b65893acc59052c1 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.826051 8562 leader_election.cc:290] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:55.824499 8083 leader_election.cc:343] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.826694 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet eaa41f8bf7c54673b65893acc59052c1 (table default.Table_33d362b6df3e40b68a3221d89a6b8358 [id=b4c56d84dddf453e96afc2c368def28e]) successfully deleted
I20251028 02:59:55.826920 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9feee73521df42ae89f5af5c5eacf9bd" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.827240 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9feee73521df42ae89f5af5c5eacf9bd" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 02:59:55.827283 8083 leader_election.cc:343] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.827395 8284 raft_consensus.cc:3060] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.828377 8284 raft_consensus.cc:2468] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.828385 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ebc0e6d526364d4ba37e443d1e681fbc
I20251028 02:59:55.828681 8083 leader_election.cc:304] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:55.828742 7961 catalog_manager.cc:6719] ReplaceTablet: tablet ebc0e6d526364d4ba37e443d1e681fbc deleted and replaced by tablet 6599ec60c3004e349e7e576e4d13c4fe
I20251028 02:59:55.829005 8550 ts_tablet_manager.cc:1950] T a198db14385543d99afb1881b0fe2409 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.829049 8255 tablet_service.cc:1552] Processing DeleteTablet for tablet ebc0e6d526364d4ba37e443d1e681fbc with delete_type TABLET_DATA_DELETED (replaced by tablet 6599ec60c3004e349e7e576e4d13c4fe) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.829221 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet ebc0e6d526364d4ba37e443d1e681fbc with delete_type TABLET_DATA_DELETED (replaced by tablet 6599ec60c3004e349e7e576e4d13c4fe) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.829509 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a198db14385543d99afb1881b0fe2409 (table default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]) successfully deleted
I20251028 02:59:55.829594 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet ebc0e6d526364d4ba37e443d1e681fbc with delete_type TABLET_DATA_DELETED (replaced by tablet 6599ec60c3004e349e7e576e4d13c4fe) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.829856 8550 tablet_replica.cc:333] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.830035 8550 raft_consensus.cc:2243] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.829841 8552 ts_tablet_manager.cc:1929] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.830245 8550 raft_consensus.cc:2272] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.830443 8552 log.cc:1199] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/67c6ceefa3294af8b68150609f46c117
I20251028 02:59:55.830679 8562 raft_consensus.cc:2804] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.830766 8552 ts_tablet_manager.cc:1950] T 67c6ceefa3294af8b68150609f46c117 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.830906 8562 raft_consensus.cc:697] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:55.831542 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 67c6ceefa3294af8b68150609f46c117 (table default.Table_f9254be0618c49f8a6a36fb9fc10354a [id=86c2f59df3dd4d2195d18835dceff79b]) successfully deleted
I20251028 02:59:55.831696 8552 tablet_replica.cc:333] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:55.831842 8552 raft_consensus.cc:2243] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
W20251028 02:59:55.831887 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 55e72a7f8aa842049da7aeaaa7001a28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55e72a7f8aa842049da7aeaaa7001a28 already in progress: creating tablet
W20251028 02:59:55.832022 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 6dfc5240893546d7a6a5397cd08044dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6dfc5240893546d7a6a5397cd08044dd already in progress: creating tablet
I20251028 02:59:55.832069 8552 raft_consensus.cc:2272] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:55.832175 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet aaaec176a3884a79a55bb74397a7dc9c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aaaec176a3884a79a55bb74397a7dc9c already in progress: creating tablet
I20251028 02:59:55.832391 8551 tablet_replica.cc:333] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.832469 8552 ts_tablet_manager.cc:1916] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.832546 8562 consensus_queue.cc:237] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.833036 8196 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:55.833165 9366 consensus_queue.cc:1048] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:55.833191 8550 ts_tablet_manager.cc:1916] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.837651 8550 ts_tablet_manager.cc:1929] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:55.837958 8550 log.cc:1199] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/67c6ceefa3294af8b68150609f46c117
I20251028 02:59:55.838546 8550 ts_tablet_manager.cc:1950] T 67c6ceefa3294af8b68150609f46c117 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.838616 9467 raft_consensus.cc:493] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.838723 9467 raft_consensus.cc:515] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.838891 8562 raft_consensus.cc:493] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.838973 8562 raft_consensus.cc:515] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.839360 8562 leader_election.cc:290] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.839491 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8a81c8bc182442bd8596251b99f90a2d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:55.839825 8218 leader_election.cc:343] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.840215 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 67c6ceefa3294af8b68150609f46c117 (table default.Table_f9254be0618c49f8a6a36fb9fc10354a [id=86c2f59df3dd4d2195d18835dceff79b]) successfully deleted
W20251028 02:59:55.840412 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 55e72a7f8aa842049da7aeaaa7001a28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55e72a7f8aa842049da7aeaaa7001a28 already in progress: creating tablet
W20251028 02:59:55.840566 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6dfc5240893546d7a6a5397cd08044dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6dfc5240893546d7a6a5397cd08044dd already in progress: creating tablet
I20251028 02:59:55.840741 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4416e1e10f53401cbddc525d9556d107" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.840879 8283 raft_consensus.cc:2468] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.841002 8550 tablet_replica.cc:333] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.841176 8083 leader_election.cc:304] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.841426 8550 raft_consensus.cc:2243] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.841775 8550 pending_rounds.cc:70] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:55.841823 9467 leader_election.cc:290] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.835675 8545 tablet_bootstrap.cc:492] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.842012 8545 ts_tablet_manager.cc:1403] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.052s user 0.000s sys 0.002s
I20251028 02:59:55.842025 8550 pending_rounds.cc:77] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597140733648896 op_type: NO_OP noop_request { }
I20251028 02:59:55.842247 8550 raft_consensus.cc:2889] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:55.842577 8545 raft_consensus.cc:359] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.842685 8545 raft_consensus.cc:385] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.842716 8545 raft_consensus.cc:740] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.842859 8545 consensus_queue.cc:260] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.842916 9190 raft_consensus.cc:493] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.842878 8562 raft_consensus.cc:2804] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.843003 9190 raft_consensus.cc:515] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.843163 8545 ts_tablet_manager.cc:1434] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:55.843235 8562 raft_consensus.cc:493] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.843393 8545 tablet_bootstrap.cc:492] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.843468 9190 leader_election.cc:290] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.843416 8562 raft_consensus.cc:3060] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.844475 8545 tablet_bootstrap.cc:654] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.840461 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4416e1e10f53401cbddc525d9556d107" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.845062 8416 raft_consensus.cc:2468] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:55.841737 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8a81c8bc182442bd8596251b99f90a2d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:55.845129 8562 raft_consensus.cc:515] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.845273 8420 raft_consensus.cc:2468] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:55.845548 8220 leader_election.cc:304] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:55.845925 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ac782a9f0b564667a7b20f1e328f2572" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:55.845938 8546 tablet_bootstrap.cc:492] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.846065 8283 raft_consensus.cc:2468] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:55.846105 8546 ts_tablet_manager.cc:1403] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.089s user 0.002s sys 0.000s
I20251028 02:59:55.846371 8356 leader_election.cc:304] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:55.846583 8938 raft_consensus.cc:2804] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.846637 8938 raft_consensus.cc:493] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.846678 8938 raft_consensus.cc:3060] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.846628 8546 raft_consensus.cc:359] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.846674 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ac782a9f0b564667a7b20f1e328f2572" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:55.846731 8546 raft_consensus.cc:385] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.846777 8546 raft_consensus.cc:740] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.846781 8147 raft_consensus.cc:2468] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:55.846911 8546 consensus_queue.cc:260] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.847198 8546 ts_tablet_manager.cc:1434] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.847431 8546 tablet_bootstrap.cc:492] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.847462 8551 raft_consensus.cc:2243] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:55.847646 8551 pending_rounds.cc:70] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:55.847676 8551 pending_rounds.cc:77] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597139722465280 op_type: NO_OP noop_request { }
I20251028 02:59:55.847728 8551 raft_consensus.cc:2889] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:55.847772 8551 raft_consensus.cc:2272] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.847906 9467 raft_consensus.cc:2804] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:55.847951 9467 raft_consensus.cc:493] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:55.847987 9467 raft_consensus.cc:3060] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.847913 8938 raft_consensus.cc:515] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.848345 8938 leader_election.cc:290] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:55.848604 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ac782a9f0b564667a7b20f1e328f2572" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:55.848701 8147 raft_consensus.cc:3060] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.849125 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_326908ab91a545829b75b06816614771" } modify_external_catalogs: true
I20251028 02:59:55.849248 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_326908ab91a545829b75b06816614771" } modify_external_catalogs: true
I20251028 02:59:55.849649 8546 tablet_bootstrap.cc:654] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.849637 8147 raft_consensus.cc:2468] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:55.850065 8354 leader_election.cc:304] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.851516 8562 leader_election.cc:290] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.851718 8550 raft_consensus.cc:2272] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.852337 9190 raft_consensus.cc:2804] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.852456 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ac782a9f0b564667a7b20f1e328f2572" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.852504 9190 raft_consensus.cc:697] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:55.852567 8283 raft_consensus.cc:3060] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.852667 9190 consensus_queue.cc:237] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.853008 8550 ts_tablet_manager.cc:1916] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.853550 8283 raft_consensus.cc:2468] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:55.854410 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4416e1e10f53401cbddc525d9556d107" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.854517 8420 raft_consensus.cc:3060] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.855479 8420 raft_consensus.cc:2468] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.856577 8545 tablet_bootstrap.cc:492] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.856741 8545 ts_tablet_manager.cc:1403] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20251028 02:59:55.856940 8083 leader_election.cc:304] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:55.857642 8562 raft_consensus.cc:2804] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.858125 8551 ts_tablet_manager.cc:1916] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.857937 8545 raft_consensus.cc:359] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.858043 8562 raft_consensus.cc:697] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:55.858235 8545 raft_consensus.cc:385] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.858366 8545 raft_consensus.cc:740] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.858577 8562 consensus_queue.cc:237] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.858750 8545 consensus_queue.cc:260] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.858940 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4416e1e10f53401cbddc525d9556d107" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:55.859056 8283 raft_consensus.cc:3060] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:55.859930 8545 ts_tablet_manager.cc:1434] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251028 02:59:55.860153 8283 raft_consensus.cc:2468] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:55.860724 8545 tablet_bootstrap.cc:492] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.863088 8545 tablet_bootstrap.cc:654] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.865594 9366 consensus_queue.cc:1048] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:55.866990 8550 ts_tablet_manager.cc:1929] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.867472 8550 log.cc:1199] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/1ef353f5e715452eae1f1c191fe79d1d
I20251028 02:59:55.868639 8550 ts_tablet_manager.cc:1950] T 1ef353f5e715452eae1f1c191fe79d1d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.869545 9467 raft_consensus.cc:515] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.869642 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 1ef353f5e715452eae1f1c191fe79d1d (table default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]) successfully deleted
I20251028 02:59:55.870345 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8a81c8bc182442bd8596251b99f90a2d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:55.870469 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f08a6e30211a4d59934c36c7a26ac983 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f08a6e30211a4d59934c36c7a26ac983 already in progress: creating tablet
I20251028 02:59:55.870644 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8a81c8bc182442bd8596251b99f90a2d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:55.870762 8416 raft_consensus.cc:3060] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:55.870860 8218 leader_election.cc:343] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:55.871021 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet aaaec176a3884a79a55bb74397a7dc9c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aaaec176a3884a79a55bb74397a7dc9c already in progress: creating tablet
I20251028 02:59:55.871271 8550 tablet_replica.cc:333] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.871610 8550 raft_consensus.cc:2243] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.871898 8416 raft_consensus.cc:2468] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:55.871907 8550 raft_consensus.cc:2272] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.872081 8556 consensus_queue.cc:1048] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:55.872321 8220 leader_election.cc:304] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:55.872459 9467 leader_election.cc:290] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.872572 8550 ts_tablet_manager.cc:1916] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.872613 9467 raft_consensus.cc:2804] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:55.875144 9467 raft_consensus.cc:697] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:55.875914 8552 ts_tablet_manager.cc:1929] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20251028 02:59:55.876540 8551 ts_tablet_manager.cc:1929] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:55.876689 8551 log.cc:1199] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ebc0e6d526364d4ba37e443d1e681fbc
I20251028 02:59:55.877029 8551 ts_tablet_manager.cc:1950] T ebc0e6d526364d4ba37e443d1e681fbc P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.877140 8552 log.cc:1199] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/1c81698dcbe14d5ba61471e7249705db
I20251028 02:59:55.877476 8552 ts_tablet_manager.cc:1950] T 1c81698dcbe14d5ba61471e7249705db P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:55.878048 8550 ts_tablet_manager.cc:1929] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20251028 02:59:55.878244 8550 log.cc:1199] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/1c81698dcbe14d5ba61471e7249705db
I20251028 02:59:55.875397 9467 consensus_queue.cc:237] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.878855 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 1c81698dcbe14d5ba61471e7249705db (table default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52]) successfully deleted
I20251028 02:59:55.879467 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ebc0e6d526364d4ba37e443d1e681fbc (table default.Table_4f4721dca71f432b82e320f8fa9b0cf5 [id=bc3f005bc9424552831e2ecca5d049a7]) successfully deleted
I20251028 02:59:55.880043 8550 ts_tablet_manager.cc:1950] T 1c81698dcbe14d5ba61471e7249705db P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:55.880311 8544 tablet_bootstrap.cc:492] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.880394 8544 ts_tablet_manager.cc:1403] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.000s
I20251028 02:59:55.880642 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 1c81698dcbe14d5ba61471e7249705db (table default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52]) successfully deleted
I20251028 02:59:55.880968 8544 raft_consensus.cc:359] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.881083 8544 raft_consensus.cc:385] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.881112 8544 raft_consensus.cc:740] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.881217 8544 consensus_queue.cc:260] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.881469 8544 ts_tablet_manager.cc:1434] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.884150 7963 catalog_manager.cc:5649] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
W20251028 02:59:55.884807 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet f08a6e30211a4d59934c36c7a26ac983 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f08a6e30211a4d59934c36c7a26ac983 already in progress: creating tablet
I20251028 02:59:55.885149 8552 tablet_replica.cc:333] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 02:59:55.885164 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9bf28d9bd3bd462f864fc4f20468e605 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9bf28d9bd3bd462f864fc4f20468e605 already in progress: creating tablet
I20251028 02:59:55.885488 8552 raft_consensus.cc:2243] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.885661 8552 raft_consensus.cc:2272] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.885622 8550 tablet_replica.cc:333] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:55.887389 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet a8f43f58ec624ca4bb12835f2e0eecbf (table default.Table_e2689b73a8454c8dbeab470093ad1463 [id=2f8c8bd6c79d43c894e6356c6e333e7c]). Expected version 1 got 0
I20251028 02:59:55.887029 8550 raft_consensus.cc:2243] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.890414 8550 raft_consensus.cc:2272] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.893385 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4817e3b21c474b2589d30ef9823032d2
I20251028 02:59:55.893814 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 9583046f549d411a9e341560f428381c from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:55.893967 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 4817e3b21c474b2589d30ef9823032d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.893910 8544 tablet_bootstrap.cc:492] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:55.894827 8025 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4817e3b21c474b2589d30ef9823032d2
I20251028 02:59:55.894963 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 4817e3b21c474b2589d30ef9823032d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.895931 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 4817e3b21c474b2589d30ef9823032d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.896575 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5155389485d54c1b82c45eb65364620f
I20251028 02:59:55.896572 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 4817e3b21c474b2589d30ef9823032d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.897109 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a8f43f58ec624ca4bb12835f2e0eecbf
I20251028 02:59:55.898473 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 5155389485d54c1b82c45eb65364620f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.898802 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet a8f43f58ec624ca4bb12835f2e0eecbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.898923 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 5155389485d54c1b82c45eb65364620f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.899240 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_65ab6aba20224249bcf1b91e6c9c65dd" } new_table_name: "default.Table_dbd811be59344e8587fff44779930d85" modify_external_catalogs: true
I20251028 02:59:55.899622 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet a8f43f58ec624ca4bb12835f2e0eecbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.900717 7960 catalog_manager.cc:5649] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:55.901346 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet b993932fd567416984ad6d28f8cadad6
I20251028 02:59:55.901628 7965 catalog_manager.cc:6719] ReplaceTablet: tablet b993932fd567416984ad6d28f8cadad6 deleted and replaced by tablet 5c1994484abf4b6fbbc3f47527558cdb
I20251028 02:59:55.902400 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet b993932fd567416984ad6d28f8cadad6 with delete_type TABLET_DATA_DELETED (replaced by tablet 5c1994484abf4b6fbbc3f47527558cdb) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:55.903148 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4817e3b21c474b2589d30ef9823032d2 because the tablet was not found. No further retry: Not found: Tablet not found: 4817e3b21c474b2589d30ef9823032d2
I20251028 02:59:55.903164 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet b993932fd567416984ad6d28f8cadad6 with delete_type TABLET_DATA_DELETED (replaced by tablet 5c1994484abf4b6fbbc3f47527558cdb) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:55.903403 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4817e3b21c474b2589d30ef9823032d2 because the tablet was not found. No further retry: Not found: Tablet not found: 4817e3b21c474b2589d30ef9823032d2
I20251028 02:59:55.904160 8550 ts_tablet_manager.cc:1916] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.905879 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet cbecb6472dc54989beca91ca7bcb0b57
W20251028 02:59:55.905925 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 5155389485d54c1b82c45eb65364620f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5155389485d54c1b82c45eb65364620f already in progress: creating tablet
I20251028 02:59:55.895510 8255 tablet_service.cc:1552] Processing DeleteTablet for tablet 4817e3b21c474b2589d30ef9823032d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.898643 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 5155389485d54c1b82c45eb65364620f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.901773 8553 consensus_queue.cc:1048] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:55.902562 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet a8f43f58ec624ca4bb12835f2e0eecbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.903870 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet b993932fd567416984ad6d28f8cadad6 with delete_type TABLET_DATA_DELETED (replaced by tablet 5c1994484abf4b6fbbc3f47527558cdb) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.906347 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 19efc09c08b540a3b42c4107de5acf73
I20251028 02:59:55.904176 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 4817e3b21c474b2589d30ef9823032d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.906742 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet df60914093f444fda88c13a132587513
I20251028 02:59:55.907006 7962 catalog_manager.cc:6719] ReplaceTablet: tablet df60914093f444fda88c13a132587513 deleted and replaced by tablet e3b33c4d189247a89878b6a8e1fcbac0
I20251028 02:59:55.909572 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 19efc09c08b540a3b42c4107de5acf73 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.909711 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet df60914093f444fda88c13a132587513 with delete_type TABLET_DATA_DELETED (replaced by tablet e3b33c4d189247a89878b6a8e1fcbac0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.909916 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet cbecb6472dc54989beca91ca7bcb0b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.910976 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9583046f549d411a9e341560f428381c
I20251028 02:59:55.911078 8553 raft_consensus.cc:493] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:55.911161 8553 raft_consensus.cc:515] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.911252 7961 catalog_manager.cc:6719] ReplaceTablet: tablet 9583046f549d411a9e341560f428381c deleted and replaced by tablet ba64524142934af08658e434e7fb08ed
I20251028 02:59:55.911561 8553 leader_election.cc:290] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:55.911839 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce64e4407ad048e8add422d77de6c211" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:55.912096 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce64e4407ad048e8add422d77de6c211" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:55.912207 8218 leader_election.cc:343] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:55.912420 8220 leader_election.cc:343] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:55.912480 8220 leader_election.cc:304] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:55.912736 8553 raft_consensus.cc:2749] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:55.912793 8552 ts_tablet_manager.cc:1916] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.911851 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 9583046f549d411a9e341560f428381c with delete_type TABLET_DATA_DELETED (replaced by tablet ba64524142934af08658e434e7fb08ed) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.913982 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet cbecb6472dc54989beca91ca7bcb0b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.914158 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 9583046f549d411a9e341560f428381c with delete_type TABLET_DATA_DELETED (replaced by tablet ba64524142934af08658e434e7fb08ed) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.914177 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet df60914093f444fda88c13a132587513 with delete_type TABLET_DATA_DELETED (replaced by tablet e3b33c4d189247a89878b6a8e1fcbac0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.914502 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet cbecb6472dc54989beca91ca7bcb0b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.914530 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 19efc09c08b540a3b42c4107de5acf73 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.914633 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet 19efc09c08b540a3b42c4107de5acf73 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.914677 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet df60914093f444fda88c13a132587513 with delete_type TABLET_DATA_DELETED (replaced by tablet e3b33c4d189247a89878b6a8e1fcbac0) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.914721 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 9583046f549d411a9e341560f428381c with delete_type TABLET_DATA_DELETED (replaced by tablet ba64524142934af08658e434e7fb08ed) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:55.915442 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45572222af854b34a5f26a00f8ad9615 consensus state has no leader
I20251028 02:59:55.918669 8551 tablet_replica.cc:333] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.919282 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 9feee73521df42ae89f5af5c5eacf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.919353 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a81c8bc182442bd8596251b99f90a2d with delete_type TABLET_DATA_DELETED (replaced by tablet c1470187cdc74716b727052ea2b0b4a0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.919505 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 4416e1e10f53401cbddc525d9556d107 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.919638 8551 raft_consensus.cc:2243] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.919821 8551 raft_consensus.cc:2272] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.922384 8551 ts_tablet_manager.cc:1916] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.922425 8544 tablet_bootstrap.cc:654] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.922497 8395 tablet_service.cc:1505] Processing CreateTablet for tablet c0fbcdef1007456997db9fcf610f1e00 (DEFAULT_TABLE table=default.Table_debfc8aa483543dcbc3c9e37d7a16db6 [id=6e7f7ac227db47c98e3a115dddc9ceff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.923059 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0fbcdef1007456997db9fcf610f1e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.923115 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_861351bbb646467db2388b94e51e0d7e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.923564 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_861351bbb646467db2388b94e51e0d7e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.925100 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 8a81c8bc182442bd8596251b99f90a2d with delete_type TABLET_DATA_DELETED (replaced by tablet c1470187cdc74716b727052ea2b0b4a0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.925181 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 9feee73521df42ae89f5af5c5eacf9bd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.925449 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 4416e1e10f53401cbddc525d9556d107 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:53 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.925535 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8924bd5725814f70bc3834b45c618074" } new_table_name: "default.Table_6088be97ba4b4513b640257c2d69675a" modify_external_catalogs: true
I20251028 02:59:55.925638 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_eca57c87ce0240a0844cc210e4dbdd85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.925994 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eca57c87ce0240a0844cc210e4dbdd85 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.927253 8551 ts_tablet_manager.cc:1929] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.927342 8551 log.cc:1199] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a8f43f58ec624ca4bb12835f2e0eecbf
I20251028 02:59:55.927558 8551 ts_tablet_manager.cc:1950] T a8f43f58ec624ca4bb12835f2e0eecbf P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.928435 8551 tablet_replica.cc:333] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.928573 8551 raft_consensus.cc:2243] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.928665 8551 raft_consensus.cc:2272] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.929018 8551 ts_tablet_manager.cc:1916] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.929665 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c149f5a7a9164664a7a699ee42b93697"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.930097 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c149f5a7a9164664a7a699ee42b93697 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.932214 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a8f43f58ec624ca4bb12835f2e0eecbf (table default.Table_e2689b73a8454c8dbeab470093ad1463 [id=2f8c8bd6c79d43c894e6356c6e333e7c]) successfully deleted
W20251028 02:59:55.932354 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet b993932fd567416984ad6d28f8cadad6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b993932fd567416984ad6d28f8cadad6 already in progress: creating tablet
W20251028 02:59:55.932459 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet cbecb6472dc54989beca91ca7bcb0b57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cbecb6472dc54989beca91ca7bcb0b57 already in progress: creating tablet
W20251028 02:59:55.932564 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 9583046f549d411a9e341560f428381c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9583046f549d411a9e341560f428381c already in progress: creating tablet
W20251028 02:59:55.932682 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet df60914093f444fda88c13a132587513 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df60914093f444fda88c13a132587513 already in progress: creating tablet
I20251028 02:59:55.933359 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet aa0776d1a8ef43449309221b86ed3923 from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:55.933468 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6f6f58ddaa6940248065a05b75996112 consensus state has no leader
I20251028 02:59:55.942516 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet aa0776d1a8ef43449309221b86ed3923
I20251028 02:59:55.942531 8551 ts_tablet_manager.cc:1929] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.942618 8551 log.cc:1199] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/19efc09c08b540a3b42c4107de5acf73
I20251028 02:59:55.942842 7969 catalog_manager.cc:6719] ReplaceTablet: tablet aa0776d1a8ef43449309221b86ed3923 deleted and replaced by tablet ad926e9ee0044c62a7abbfca959a4f56
I20251028 02:59:55.942848 8551 ts_tablet_manager.cc:1950] T 19efc09c08b540a3b42c4107de5acf73 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.944058 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet aa0776d1a8ef43449309221b86ed3923 with delete_type TABLET_DATA_DELETED (replaced by tablet ad926e9ee0044c62a7abbfca959a4f56) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.946411 8469 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:55.947857 8126 tablet_service.cc:1505] Processing CreateTablet for tablet c0fbcdef1007456997db9fcf610f1e00 (DEFAULT_TABLE table=default.Table_debfc8aa483543dcbc3c9e37d7a16db6 [id=6e7f7ac227db47c98e3a115dddc9ceff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.947976 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet aa0776d1a8ef43449309221b86ed3923 with delete_type TABLET_DATA_DELETED (replaced by tablet ad926e9ee0044c62a7abbfca959a4f56) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.948158 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0fbcdef1007456997db9fcf610f1e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.948174 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet aa0776d1a8ef43449309221b86ed3923 with delete_type TABLET_DATA_DELETED (replaced by tablet ad926e9ee0044c62a7abbfca959a4f56) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.949072 8333 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:55.949402 8258 tablet_service.cc:1505] Processing CreateTablet for tablet c0fbcdef1007456997db9fcf610f1e00 (DEFAULT_TABLE table=default.Table_debfc8aa483543dcbc3c9e37d7a16db6 [id=6e7f7ac227db47c98e3a115dddc9ceff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.951059 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 0fcabc4a2fd44043be5451046ff5bd97 (DEFAULT_TABLE table=default.Table_31e973dfaa9b49f2bed3e9dfd11f7ae9 [id=f6bf7ec7e30f4c469ca1adde25fee2d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.951395 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fcabc4a2fd44043be5451046ff5bd97. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.951601 8551 tablet_replica.cc:333] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.951592 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 19efc09c08b540a3b42c4107de5acf73 (table default.Table_4b9a111a46cd448cac4abf954888e213 [id=097df645a88549b68a7c84c94f158190]) successfully deleted
I20251028 02:59:55.951773 8551 raft_consensus.cc:2243] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.951900 8551 raft_consensus.cc:2272] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.952623 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 0fcabc4a2fd44043be5451046ff5bd97 (DEFAULT_TABLE table=default.Table_31e973dfaa9b49f2bed3e9dfd11f7ae9 [id=f6bf7ec7e30f4c469ca1adde25fee2d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.952941 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fcabc4a2fd44043be5451046ff5bd97. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.953807 8546 tablet_bootstrap.cc:492] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.953913 8546 ts_tablet_manager.cc:1403] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.107s user 0.003s sys 0.000s
I20251028 02:59:55.954195 8551 ts_tablet_manager.cc:1916] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.954267 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0fbcdef1007456997db9fcf610f1e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.954396 8546 raft_consensus.cc:359] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.954494 8546 raft_consensus.cc:385] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.954532 8546 raft_consensus.cc:740] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.954661 8546 consensus_queue.cc:260] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.954943 8546 ts_tablet_manager.cc:1434] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.955143 8546 tablet_bootstrap.cc:492] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:55.956151 8546 tablet_bootstrap.cc:654] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.956391 8551 ts_tablet_manager.cc:1929] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.956457 8551 log.cc:1199] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8a81c8bc182442bd8596251b99f90a2d
I20251028 02:59:55.956663 8551 ts_tablet_manager.cc:1950] T 8a81c8bc182442bd8596251b99f90a2d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.957022 8544 tablet_bootstrap.cc:492] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:55.957113 8545 tablet_bootstrap.cc:492] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:55.957122 8544 ts_tablet_manager.cc:1403] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.063s user 0.001s sys 0.001s
I20251028 02:59:55.957191 8545 ts_tablet_manager.cc:1403] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.097s user 0.003s sys 0.000s
I20251028 02:59:55.957572 8544 raft_consensus.cc:359] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.957677 8544 raft_consensus.cc:385] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.957706 8544 raft_consensus.cc:740] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.957690 8545 raft_consensus.cc:359] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.957799 8545 raft_consensus.cc:385] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:55.957831 8545 raft_consensus.cc:740] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:55.957863 8544 consensus_queue.cc:260] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.957943 8545 consensus_queue.cc:260] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.958209 8545 ts_tablet_manager.cc:1434] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:55.958406 8545 tablet_bootstrap.cc:492] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:55.961021 8552 ts_tablet_manager.cc:1929] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.961719 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 0fcabc4a2fd44043be5451046ff5bd97 (DEFAULT_TABLE table=default.Table_31e973dfaa9b49f2bed3e9dfd11f7ae9 [id=f6bf7ec7e30f4c469ca1adde25fee2d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.965000 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fcabc4a2fd44043be5451046ff5bd97. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.965274 8545 tablet_bootstrap.cc:654] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.966647 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8a81c8bc182442bd8596251b99f90a2d (table default.Table_63b74aebe1364823bb6a10f4a8942db8 [id=485d972ab1864e26ace68ba52bb671ad]) successfully deleted
I20251028 02:59:55.967032 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 73c20437695a4043a86faa9c98091e5e with delete_type TABLET_DATA_DELETED (replaced by tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:55.967252 8551 tablet_replica.cc:333] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.967403 8551 raft_consensus.cc:2243] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.967556 8551 raft_consensus.cc:2272] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.967566 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet ce64e4407ad048e8add422d77de6c211 with delete_type TABLET_DATA_DELETED (replaced by tablet a9653fc3236f4e0b812de5a859932587) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:55.967866 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f56d1ad60fea4b2f9425aa0f9280df5d" } new_table_name: "default.Table_a74b1be50de04891b89cb1445a48a40e" modify_external_catalogs: true
I20251028 02:59:55.972810 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 69e0358b593f4b55b6b4ec859e2c021d (DEFAULT_TABLE table=default.Table_1e1f686b7dda4c03b402b11bae923b8e [id=e88ffc49924340aa92a3aedd631330bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.973093 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69e0358b593f4b55b6b4ec859e2c021d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.973341 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 69e0358b593f4b55b6b4ec859e2c021d (DEFAULT_TABLE table=default.Table_1e1f686b7dda4c03b402b11bae923b8e [id=e88ffc49924340aa92a3aedd631330bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.973625 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69e0358b593f4b55b6b4ec859e2c021d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.974273 8551 ts_tablet_manager.cc:1916] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.975566 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 07d3e6c640434652a010942402c54716 (DEFAULT_TABLE table=default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.975847 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d3e6c640434652a010942402c54716. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.976425 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 07d3e6c640434652a010942402c54716 (DEFAULT_TABLE table=default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.976792 8254 tablet_service.cc:1505] Processing CreateTablet for tablet 07d3e6c640434652a010942402c54716 (DEFAULT_TABLE table=default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.977082 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d3e6c640434652a010942402c54716. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.977276 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 69e0358b593f4b55b6b4ec859e2c021d (DEFAULT_TABLE table=default.Table_1e1f686b7dda4c03b402b11bae923b8e [id=e88ffc49924340aa92a3aedd631330bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.977567 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69e0358b593f4b55b6b4ec859e2c021d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.978215 8550 ts_tablet_manager.cc:1929] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.979363 8551 ts_tablet_manager.cc:1929] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.979550 8551 log.cc:1199] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9feee73521df42ae89f5af5c5eacf9bd
I20251028 02:59:55.979362 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d3e6c640434652a010942402c54716. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.979008 8550 log.cc:1199] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/eaa41f8bf7c54673b65893acc59052c1
I20251028 02:59:55.979898 8551 ts_tablet_manager.cc:1950] T 9feee73521df42ae89f5af5c5eacf9bd P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:55.980108 8544 ts_tablet_manager.cc:1434] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.023s user 0.001s sys 0.000s
I20251028 02:59:55.980508 8551 tablet_replica.cc:333] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.980644 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9feee73521df42ae89f5af5c5eacf9bd (table default.Table_8cae8460736e4dcb9108757c62255907 [id=8d264830878e487daa057d286f1e3e4f]) successfully deleted
I20251028 02:59:55.980695 8551 raft_consensus.cc:2243] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.980902 8551 raft_consensus.cc:2272] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.982230 8400 tablet_service.cc:1505] Processing CreateTablet for tablet ba40ba2c066e4c94885a595fd1c4445e (DEFAULT_TABLE table=default.Table_215f8629159d41069edc7ce48ed97ae7 [id=25d0c2f2faef473b94ce2621281c08b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.982466 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba40ba2c066e4c94885a595fd1c4445e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.983212 8551 ts_tablet_manager.cc:1916] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.985301 8551 ts_tablet_manager.cc:1929] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:55.986217 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0d1a3e0b91aa41dcb3264e04169fdd5e" } new_table_name: "default.Table_745438bc0c014b13ab3381877a959e1e" modify_external_catalogs: true
I20251028 02:59:55.986264 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5bcebe955d05492e8055eec721f46a57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.986653 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5bcebe955d05492e8055eec721f46a57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.987344 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_46656faddd3141adbdd28b70154be5f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.987459 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d9c99a82fc094c599c4b1b85f52b6be9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:55.987576 8551 log.cc:1199] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4416e1e10f53401cbddc525d9556d107
W20251028 02:59:55.987708 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d9c99a82fc094c599c4b1b85f52b6be9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.987829 8551 ts_tablet_manager.cc:1950] T 4416e1e10f53401cbddc525d9556d107 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 02:59:55.987708 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46656faddd3141adbdd28b70154be5f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.988497 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4416e1e10f53401cbddc525d9556d107 (table default.Table_322bdb2bdb474e39937bd5b0ecf1556e [id=2b814ef3eb474d1fa873eaeff4fa1799]) successfully deleted
I20251028 02:59:55.988463 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4debdeef26b14d9b93a878210eaba4c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:55.988791 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4debdeef26b14d9b93a878210eaba4c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:55.989250 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f2b0692abe27469a8f0728978e62ae26" } new_table_name: "default.Table_9e950f6a07524cb0b5a0d3d967668ad6" modify_external_catalogs: true
I20251028 02:59:55.990000 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet ce64e4407ad048e8add422d77de6c211 with delete_type TABLET_DATA_DELETED (replaced by tablet a9653fc3236f4e0b812de5a859932587) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:55.993754 8544 tablet_bootstrap.cc:492] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 02:59:55.994325 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e3b33c4d189247a89878b6a8e1fcbac0 has no consensus state
W20251028 02:59:55.995123 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet aa0776d1a8ef43449309221b86ed3923 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa0776d1a8ef43449309221b86ed3923 already in progress: creating tablet
I20251028 02:59:55.995410 8551 tablet_replica.cc:333] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:55.996426 8544 tablet_bootstrap.cc:654] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:55.997521 8551 raft_consensus.cc:2243] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:55.997704 8551 raft_consensus.cc:2272] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:55.998322 8551 ts_tablet_manager.cc:1916] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:55.998710 8126 tablet_service.cc:1505] Processing CreateTablet for tablet ba40ba2c066e4c94885a595fd1c4445e (DEFAULT_TABLE table=default.Table_215f8629159d41069edc7ce48ed97ae7 [id=25d0c2f2faef473b94ce2621281c08b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:55.998986 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba40ba2c066e4c94885a595fd1c4445e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:55.999194 8546 tablet_bootstrap.cc:492] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:55.999318 8546 ts_tablet_manager.cc:1403] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.000s
I20251028 02:59:55.999874 8546 raft_consensus.cc:359] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:55.999986 8546 raft_consensus.cc:385] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.000020 8546 raft_consensus.cc:740] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.000145 8546 consensus_queue.cc:260] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.000456 8546 ts_tablet_manager.cc:1434] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.000660 8546 tablet_bootstrap.cc:492] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:56.002022 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5c1994484abf4b6fbbc3f47527558cdb has no consensus state
I20251028 02:59:56.003093 8545 tablet_bootstrap.cc:492] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.003198 8545 ts_tablet_manager.cc:1403] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.045s user 0.003s sys 0.000s
I20251028 02:59:56.003742 8545 raft_consensus.cc:359] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.003866 8545 raft_consensus.cc:385] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.003901 8545 raft_consensus.cc:740] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.004020 8545 consensus_queue.cc:260] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.004304 8545 ts_tablet_manager.cc:1434] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:56.004518 8545 tablet_bootstrap.cc:492] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.005650 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_327424a7e56243ae97d58c0f6fd560fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.006103 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_327424a7e56243ae97d58c0f6fd560fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.008000 8545 tablet_bootstrap.cc:654] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.010293 8550 ts_tablet_manager.cc:1950] T eaa41f8bf7c54673b65893acc59052c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.010550 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_cf55a8b4734749d0b0ccc0230c8d81a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.010953 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf55a8b4734749d0b0ccc0230c8d81a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:56.011971 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a808098c70b54ad3a0230352af0b743c consensus state has no leader
I20251028 02:59:56.013742 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5384863d743c4b8baf90bc97055948a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.014104 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5384863d743c4b8baf90bc97055948a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.014990 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 68ab2fe9132441c1834aea22561f02e9 (DEFAULT_TABLE table=default.Table_b50e3ba3bc464d54ba1466ca077c9334 [id=520d66b876db4f25915642dd3092f1cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.015344 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68ab2fe9132441c1834aea22561f02e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.016171 8546 tablet_bootstrap.cc:654] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.018608 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet eaa41f8bf7c54673b65893acc59052c1 (table default.Table_33d362b6df3e40b68a3221d89a6b8358 [id=b4c56d84dddf453e96afc2c368def28e]) successfully deleted
I20251028 02:59:56.018790 8551 ts_tablet_manager.cc:1929] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.018872 8551 log.cc:1199] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ce64e4407ad048e8add422d77de6c211
I20251028 02:59:56.019441 8551 ts_tablet_manager.cc:1950] T ce64e4407ad048e8add422d77de6c211 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:56.019423 8254 tablet_service.cc:1505] Processing CreateTablet for tablet 68ab2fe9132441c1834aea22561f02e9 (DEFAULT_TABLE table=default.Table_b50e3ba3bc464d54ba1466ca077c9334 [id=520d66b876db4f25915642dd3092f1cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.019721 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68ab2fe9132441c1834aea22561f02e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.020097 8550 tablet_replica.cc:333] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.020413 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ce64e4407ad048e8add422d77de6c211 (table default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]) successfully deleted
I20251028 02:59:56.020715 8550 raft_consensus.cc:2243] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.021698 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 2bf100407b034626b4a9f7e2340f9cf0 (DEFAULT_TABLE table=default.Table_3d3d9ba2e032453fbbc6b2635d1b2203 [id=51fb1b3116c14113b2e14ced3ac17262]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.021947 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bf100407b034626b4a9f7e2340f9cf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.022223 8550 raft_consensus.cc:2272] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.022609 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 40a0431501d348b19ec89fe692741ce5 (DEFAULT_TABLE table=default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.022876 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40a0431501d348b19ec89fe692741ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.022905 8550 ts_tablet_manager.cc:1916] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.024298 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 68ab2fe9132441c1834aea22561f02e9 (DEFAULT_TABLE table=default.Table_b50e3ba3bc464d54ba1466ca077c9334 [id=520d66b876db4f25915642dd3092f1cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.024616 8258 tablet_service.cc:1505] Processing CreateTablet for tablet ba40ba2c066e4c94885a595fd1c4445e (DEFAULT_TABLE table=default.Table_215f8629159d41069edc7ce48ed97ae7 [id=25d0c2f2faef473b94ce2621281c08b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.024971 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 40a0431501d348b19ec89fe692741ce5 (DEFAULT_TABLE table=default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.020825 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 40a0431501d348b19ec89fe692741ce5 (DEFAULT_TABLE table=default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.025318 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40a0431501d348b19ec89fe692741ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.025403 8254 tablet_service.cc:1505] Processing CreateTablet for tablet 2bf100407b034626b4a9f7e2340f9cf0 (DEFAULT_TABLE table=default.Table_3d3d9ba2e032453fbbc6b2635d1b2203 [id=51fb1b3116c14113b2e14ced3ac17262]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.025514 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40a0431501d348b19ec89fe692741ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.025836 8545 tablet_bootstrap.cc:492] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.025930 8545 ts_tablet_manager.cc:1403] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.001s
I20251028 02:59:56.026471 8545 raft_consensus.cc:359] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.026593 8545 raft_consensus.cc:385] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.026643 8545 raft_consensus.cc:740] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.026820 8545 consensus_queue.cc:260] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.030162 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68ab2fe9132441c1834aea22561f02e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.030572 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bf100407b034626b4a9f7e2340f9cf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.030969 8546 tablet_bootstrap.cc:492] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.031061 8546 ts_tablet_manager.cc:1403] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.002s
I20251028 02:59:56.031912 8546 raft_consensus.cc:359] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.032020 8546 raft_consensus.cc:385] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.032055 8546 raft_consensus.cc:740] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.032210 8546 consensus_queue.cc:260] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.033488 8550 ts_tablet_manager.cc:1929] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.034075 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba40ba2c066e4c94885a595fd1c4445e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.034680 8546 ts_tablet_manager.cc:1434] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:56.034909 8546 tablet_bootstrap.cc:492] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.030926 8545 ts_tablet_manager.cc:1434] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 02:59:56.033838 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 2bf100407b034626b4a9f7e2340f9cf0 (DEFAULT_TABLE table=default.Table_3d3d9ba2e032453fbbc6b2635d1b2203 [id=51fb1b3116c14113b2e14ced3ac17262]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.034699 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 4817e3b21c474b2589d30ef9823032d2 (DEFAULT_TABLE table=default.Table_326908ab91a545829b75b06816614771 [id=06de831f38b74668aabbef36f08a4a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.038048 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4817e3b21c474b2589d30ef9823032d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.038493 8546 tablet_bootstrap.cc:654] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.038815 8545 tablet_bootstrap.cc:492] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.039899 8545 tablet_bootstrap.cc:654] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.040755 8550 log.cc:1199] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/309a9d10178e4984a18e10dfd11cba22
I20251028 02:59:56.041507 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 4817e3b21c474b2589d30ef9823032d2 (DEFAULT_TABLE table=default.Table_326908ab91a545829b75b06816614771 [id=06de831f38b74668aabbef36f08a4a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.041889 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4817e3b21c474b2589d30ef9823032d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.042271 8396 tablet_service.cc:1505] Processing CreateTablet for tablet f2d0dffdf5cc414ab870e0dde75916d6 (DEFAULT_TABLE table=default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.042657 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bf100407b034626b4a9f7e2340f9cf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.043387 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2d0dffdf5cc414ab870e0dde75916d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.043859 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 626c2cc2d19047b794bf16c1b078941f (DEFAULT_TABLE table=default.Table_797c6e55c14f415589c745d9bedfcb07 [id=81dbfb2d58ab45958553cf73e259cdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.044082 8550 ts_tablet_manager.cc:1950] T 309a9d10178e4984a18e10dfd11cba22 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.044109 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626c2cc2d19047b794bf16c1b078941f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.041831 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 4817e3b21c474b2589d30ef9823032d2 (DEFAULT_TABLE table=default.Table_326908ab91a545829b75b06816614771 [id=06de831f38b74668aabbef36f08a4a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.042496 8128 tablet_service.cc:1505] Processing CreateTablet for tablet f2d0dffdf5cc414ab870e0dde75916d6 (DEFAULT_TABLE table=default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.045719 8556 raft_consensus.cc:493] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.045769 8254 tablet_service.cc:1505] Processing CreateTablet for tablet f2d0dffdf5cc414ab870e0dde75916d6 (DEFAULT_TABLE table=default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.045809 8556 raft_consensus.cc:515] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.046000 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2d0dffdf5cc414ab870e0dde75916d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.046171 8556 leader_election.cc:290] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.046343 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 309a9d10178e4984a18e10dfd11cba22 (table default.Table_c6ab20b423864891ba7a399af893a205 [id=4e185be67d91416e835dfc1c32e7e1e8]) successfully deleted
W20251028 02:59:56.046682 8218 leader_election.cc:343] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 374936c130c841348e0d76cce070a295
I20251028 02:59:56.046736 8550 tablet_replica.cc:333] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.046886 8550 raft_consensus.cc:2243] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.047101 8550 raft_consensus.cc:2272] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:56.047168 8220 leader_election.cc:343] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 374936c130c841348e0d76cce070a295
I20251028 02:59:56.047247 8220 leader_election.cc:304] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.047454 8546 tablet_bootstrap.cc:492] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.047541 8546 ts_tablet_manager.cc:1403] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20251028 02:59:56.048010 8546 raft_consensus.cc:359] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.048089 8550 ts_tablet_manager.cc:1916] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.048105 8546 raft_consensus.cc:385] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.048146 8546 raft_consensus.cc:740] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.048259 8546 consensus_queue.cc:260] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.048499 8546 ts_tablet_manager.cc:1434] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:56.048619 8553 raft_consensus.cc:2749] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.049913 8546 tablet_bootstrap.cc:492] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.050022 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 0703e18d7dba4d628e3cc310be55fcf5 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.050200 8125 tablet_service.cc:1505] Processing CreateTablet for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (DEFAULT_TABLE table=default.Table_fd8850b2aff9461ca7f69c76704fb00e [id=674f76f9afed4f3f9e2c165e2479d938]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.051122 8396 tablet_service.cc:1505] Processing CreateTablet for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (DEFAULT_TABLE table=default.Table_fd8850b2aff9461ca7f69c76704fb00e [id=674f76f9afed4f3f9e2c165e2479d938]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.051460 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2d0dffdf5cc414ab870e0dde75916d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.051529 8261 tablet_service.cc:1505] Processing CreateTablet for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (DEFAULT_TABLE table=default.Table_fd8850b2aff9461ca7f69c76704fb00e [id=674f76f9afed4f3f9e2c165e2479d938]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.051826 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 626c2cc2d19047b794bf16c1b078941f (DEFAULT_TABLE table=default.Table_797c6e55c14f415589c745d9bedfcb07 [id=81dbfb2d58ab45958553cf73e259cdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.051894 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a28ff8c0ee46cb89ba7e28de2cce45. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.052066 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a28ff8c0ee46cb89ba7e28de2cce45. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.054724 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a28ff8c0ee46cb89ba7e28de2cce45. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.054744 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0e20b3e05c484394e11bbdea6a44ea with delete_type TABLET_DATA_DELETED (replaced by tablet bd50e7d4e28541de9799833c53fe7c02) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.055287 8545 tablet_bootstrap.cc:492] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.055397 8545 ts_tablet_manager.cc:1403] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20251028 02:59:56.055917 8545 raft_consensus.cc:359] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.056023 8545 raft_consensus.cc:385] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.056053 8545 raft_consensus.cc:740] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.056159 8545 consensus_queue.cc:260] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.056430 8545 ts_tablet_manager.cc:1434] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.056612 8545 tablet_bootstrap.cc:492] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.056993 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626c2cc2d19047b794bf16c1b078941f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.057737 8545 tablet_bootstrap.cc:654] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.060178 8546 tablet_bootstrap.cc:654] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.061050 8552 log.cc:1199] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/309a9d10178e4984a18e10dfd11cba22
I20251028 02:59:56.061501 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4817e3b21c474b2589d30ef9823032d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.064213 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 0703e18d7dba4d628e3cc310be55fcf5
I20251028 02:59:56.064572 7966 catalog_manager.cc:6719] ReplaceTablet: tablet 0703e18d7dba4d628e3cc310be55fcf5 deleted and replaced by tablet f62b981b9b1a44bb96bbc90f924c3b64
I20251028 02:59:56.064994 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 0703e18d7dba4d628e3cc310be55fcf5 with delete_type TABLET_DATA_DELETED (replaced by tablet f62b981b9b1a44bb96bbc90f924c3b64) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.066079 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 0703e18d7dba4d628e3cc310be55fcf5 with delete_type TABLET_DATA_DELETED (replaced by tablet f62b981b9b1a44bb96bbc90f924c3b64) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.066365 8551 tablet_replica.cc:333] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:56.066615 8551 raft_consensus.cc:2243] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.066874 8551 raft_consensus.cc:2272] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.066992 8552 ts_tablet_manager.cc:1950] T 309a9d10178e4984a18e10dfd11cba22 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:56.067674 8551 ts_tablet_manager.cc:1916] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.068074 8550 ts_tablet_manager.cc:1929] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.068164 8550 log.cc:1199] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ebc0e6d526364d4ba37e443d1e681fbc
I20251028 02:59:56.068409 8550 ts_tablet_manager.cc:1950] T ebc0e6d526364d4ba37e443d1e681fbc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.069089 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet ebc0e6d526364d4ba37e443d1e681fbc (table default.Table_4f4721dca71f432b82e320f8fa9b0cf5 [id=bc3f005bc9424552831e2ecca5d049a7]) successfully deleted
I20251028 02:59:56.070902 8545 tablet_bootstrap.cc:492] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.071000 8545 ts_tablet_manager.cc:1403] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20251028 02:59:56.071517 8545 raft_consensus.cc:359] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.071635 8545 raft_consensus.cc:385] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.071673 8545 raft_consensus.cc:740] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.071820 8545 consensus_queue.cc:260] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.072095 8545 ts_tablet_manager.cc:1434] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.072279 8545 tablet_bootstrap.cc:492] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.072386 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 0703e18d7dba4d628e3cc310be55fcf5 with delete_type TABLET_DATA_DELETED (replaced by tablet f62b981b9b1a44bb96bbc90f924c3b64) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.072693 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 626c2cc2d19047b794bf16c1b078941f (DEFAULT_TABLE table=default.Table_797c6e55c14f415589c745d9bedfcb07 [id=81dbfb2d58ab45958553cf73e259cdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.072947 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 626c2cc2d19047b794bf16c1b078941f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.073297 8545 tablet_bootstrap.cc:654] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.074077 8546 tablet_bootstrap.cc:492] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.074182 8546 ts_tablet_manager.cc:1403] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.001s
W20251028 02:59:56.074632 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4817e3b21c474b2589d30ef9823032d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4817e3b21c474b2589d30ef9823032d2 already in progress: creating tablet
I20251028 02:59:56.074730 8546 raft_consensus.cc:359] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.074869 8546 raft_consensus.cc:385] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.074908 8546 raft_consensus.cc:740] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.075038 8546 consensus_queue.cc:260] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.075354 8546 ts_tablet_manager.cc:1434] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.075554 8546 tablet_bootstrap.cc:492] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.075697 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 309a9d10178e4984a18e10dfd11cba22 (table default.Table_c6ab20b423864891ba7a399af893a205 [id=4e185be67d91416e835dfc1c32e7e1e8]) successfully deleted
I20251028 02:59:56.075940 8552 tablet_replica.cc:333] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:56.076084 8552 raft_consensus.cc:2243] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.076175 8552 raft_consensus.cc:2272] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.076571 8546 tablet_bootstrap.cc:654] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.076682 8552 ts_tablet_manager.cc:1916] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:56.079097 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4817e3b21c474b2589d30ef9823032d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4817e3b21c474b2589d30ef9823032d2 already in progress: creating tablet
I20251028 02:59:56.079460 8550 tablet_replica.cc:333] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.079667 8550 raft_consensus.cc:2243] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.079861 8550 raft_consensus.cc:2272] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.080343 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 506396e55fc242ada23d66088984f0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.080579 8550 ts_tablet_manager.cc:1916] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.080623 8551 ts_tablet_manager.cc:1929] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:56.080816 8551 log.cc:1199] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/0703e18d7dba4d628e3cc310be55fcf5
I20251028 02:59:56.081300 8551 ts_tablet_manager.cc:1950] T 0703e18d7dba4d628e3cc310be55fcf5 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:56.082911 8546 tablet_bootstrap.cc:492] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.083011 8546 ts_tablet_manager.cc:1403] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:56.083030 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 0703e18d7dba4d628e3cc310be55fcf5 (table default.Table_758c1adf3eb6460d9fa4e675a42507e3 [id=d4eaebd005bc42dd86d7125ef555546f]) successfully deleted
I20251028 02:59:56.083704 8546 raft_consensus.cc:359] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.083804 8546 raft_consensus.cc:385] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.083833 8546 raft_consensus.cc:740] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.083951 8546 consensus_queue.cc:260] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.084228 8546 ts_tablet_manager.cc:1434] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.084880 8544 tablet_bootstrap.cc:492] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.084987 8544 ts_tablet_manager.cc:1403] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.091s user 0.002s sys 0.000s
I20251028 02:59:56.085481 8544 raft_consensus.cc:359] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.085595 8544 raft_consensus.cc:385] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.085629 8544 raft_consensus.cc:740] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.085750 8544 consensus_queue.cc:260] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.086038 8544 ts_tablet_manager.cc:1434] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.086246 8544 tablet_bootstrap.cc:492] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.087196 8552 ts_tablet_manager.cc:1929] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.087268 8552 log.cc:1199] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4416e1e10f53401cbddc525d9556d107
I20251028 02:59:56.087533 8552 ts_tablet_manager.cc:1950] T 4416e1e10f53401cbddc525d9556d107 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:56.087708 8546 tablet_bootstrap.cc:492] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.090252 8546 tablet_bootstrap.cc:654] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.092510 8196 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:56.092589 8544 tablet_bootstrap.cc:654] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.093796 8550 ts_tablet_manager.cc:1929] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.093871 8550 log.cc:1199] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a8f43f58ec624ca4bb12835f2e0eecbf
I20251028 02:59:56.094110 8550 ts_tablet_manager.cc:1950] T a8f43f58ec624ca4bb12835f2e0eecbf P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.094825 8545 tablet_bootstrap.cc:492] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.094930 8545 ts_tablet_manager.cc:1403] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20251028 02:59:56.095508 8545 raft_consensus.cc:359] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.095613 8545 raft_consensus.cc:385] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.095640 8545 raft_consensus.cc:740] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.095778 8545 consensus_queue.cc:260] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.096062 8545 ts_tablet_manager.cc:1434] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.096249 8545 tablet_bootstrap.cc:492] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.097431 8545 tablet_bootstrap.cc:654] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.098300 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2cd00f3aa8f744bd890e7fc0efb34753" } modify_external_catalogs: true
I20251028 02:59:56.098433 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2cd00f3aa8f744bd890e7fc0efb34753" } modify_external_catalogs: true
I20251028 02:59:56.099025 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4f4721dca71f432b82e320f8fa9b0cf5" } modify_external_catalogs: true
I20251028 02:59:56.099138 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4f4721dca71f432b82e320f8fa9b0cf5" } modify_external_catalogs: true
I20251028 02:59:56.101033 8545 tablet_bootstrap.cc:492] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.101145 8545 ts_tablet_manager.cc:1403] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:56.101636 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4416e1e10f53401cbddc525d9556d107 (table default.Table_322bdb2bdb474e39937bd5b0ecf1556e [id=2b814ef3eb474d1fa873eaeff4fa1799]) successfully deleted
I20251028 02:59:56.101662 8545 raft_consensus.cc:359] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.101756 8545 raft_consensus.cc:385] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.102187 8552 tablet_replica.cc:333] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:56.102502 8552 raft_consensus.cc:2243] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.102561 8545 raft_consensus.cc:740] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.102695 8552 raft_consensus.cc:2272] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.102800 8545 consensus_queue.cc:260] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.103552 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7bfb43f76f8b46b292deee50a8f58280" } modify_external_catalogs: true
I20251028 02:59:56.103596 8552 ts_tablet_manager.cc:1916] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.103670 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7bfb43f76f8b46b292deee50a8f58280" } modify_external_catalogs: true
I20251028 02:59:56.103951 8546 tablet_bootstrap.cc:492] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.104054 8546 ts_tablet_manager.cc:1403] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 02:59:56.104547 8546 raft_consensus.cc:359] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.104656 8546 raft_consensus.cc:385] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.104692 8546 raft_consensus.cc:740] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.104823 8546 consensus_queue.cc:260] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.106676 8546 ts_tablet_manager.cc:1434] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:56.106892 8546 tablet_bootstrap.cc:492] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.107580 8545 ts_tablet_manager.cc:1434] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251028 02:59:56.108011 8546 tablet_bootstrap.cc:654] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.109689 8545 tablet_bootstrap.cc:492] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.113549 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 3f06af61617e4be9a753692e9ad4978d from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.113802 8469 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:56.114372 8552 ts_tablet_manager.cc:1929] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.114495 8552 log.cc:1199] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9bf28d9bd3bd462f864fc4f20468e605
I20251028 02:59:56.114753 8552 ts_tablet_manager.cc:1950] T 9bf28d9bd3bd462f864fc4f20468e605 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:56.114882 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bfd3f08061c3433f878612643c04387b" } modify_external_catalogs: true
I20251028 02:59:56.115002 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bfd3f08061c3433f878612643c04387b" } modify_external_catalogs: true
I20251028 02:59:56.115741 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9bf28d9bd3bd462f864fc4f20468e605 (table default.Table_ce36e2a0a4374085b1e87288b3288ab9 [id=e61fa366db844911bb80af844de41fa6]) successfully deleted
I20251028 02:59:56.115870 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_797c6e55c14f415589c745d9bedfcb07" } modify_external_catalogs: true
I20251028 02:59:56.115970 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_797c6e55c14f415589c745d9bedfcb07" } modify_external_catalogs: true
I20251028 02:59:56.115971 8552 tablet_replica.cc:333] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:56.116117 8552 raft_consensus.cc:2243] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.116223 8552 raft_consensus.cc:2272] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.116771 8552 ts_tablet_manager.cc:1916] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.118795 8545 tablet_bootstrap.cc:654] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.120291 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4ff8c974231f4b8aa18c18a05ad3e738
I20251028 02:59:56.121436 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ff8c974231f4b8aa18c18a05ad3e738 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.121623 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ff8c974231f4b8aa18c18a05ad3e738 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.121917 8551 tablet_replica.cc:333] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:56.122082 8551 raft_consensus.cc:2243] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.122212 8551 raft_consensus.cc:2272] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.122619 8551 ts_tablet_manager.cc:1916] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.111410 8544 tablet_bootstrap.cc:492] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.123826 8544 ts_tablet_manager.cc:1403] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.002s
I20251028 02:59:56.124397 8544 raft_consensus.cc:359] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.124538 8544 raft_consensus.cc:385] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.124576 8544 raft_consensus.cc:740] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.125362 8544 consensus_queue.cc:260] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.125689 8551 ts_tablet_manager.cc:1929] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:56.121687 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 4ff8c974231f4b8aa18c18a05ad3e738 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.126287 8551 log.cc:1199] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4ff8c974231f4b8aa18c18a05ad3e738
I20251028 02:59:56.126358 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_cf55a8b4734749d0b0ccc0230c8d81a6" } new_table_name: "default.Table_f623537faf00450f8b0997cb0e2ab81b" modify_external_catalogs: true
I20251028 02:59:56.126683 8551 ts_tablet_manager.cc:1950] T 4ff8c974231f4b8aa18c18a05ad3e738 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 02:59:56.125735 8083 consensus_peers.cc:597] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 19efc09c08b540a3b42c4107de5acf73. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:56.130200 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]) successfully deleted
I20251028 02:59:56.131892 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet f2d0dffdf5cc414ab870e0dde75916d6 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.127776 8545 tablet_bootstrap.cc:492] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.132511 8545 ts_tablet_manager.cc:1403] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.001s
I20251028 02:59:56.133029 8545 raft_consensus.cc:359] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.133143 8545 raft_consensus.cc:385] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.133172 8545 raft_consensus.cc:740] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.133458 8545 consensus_queue.cc:260] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.133775 8545 ts_tablet_manager.cc:1434] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.133978 8545 tablet_bootstrap.cc:492] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.135424 8546 tablet_bootstrap.cc:492] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.135527 8546 ts_tablet_manager.cc:1403] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 02:59:56.136030 8546 raft_consensus.cc:359] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.136138 8546 raft_consensus.cc:385] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.136169 8546 raft_consensus.cc:740] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.136286 8546 consensus_queue.cc:260] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.136580 8546 ts_tablet_manager.cc:1434] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.136760 8546 tablet_bootstrap.cc:492] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.137786 8546 tablet_bootstrap.cc:654] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.140442 8544 ts_tablet_manager.cc:1434] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.017s user 0.001s sys 0.000s
I20251028 02:59:56.141309 8545 tablet_bootstrap.cc:654] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.141466 9475 raft_consensus.cc:493] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.141577 9475 raft_consensus.cc:515] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.142019 9475 leader_election.cc:290] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.142724 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "df60914093f444fda88c13a132587513" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.142861 8284 raft_consensus.cc:2468] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.142978 8552 ts_tablet_manager.cc:1929] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.143046 8552 log.cc:1199] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9feee73521df42ae89f5af5c5eacf9bd
I20251028 02:59:56.143258 8552 ts_tablet_manager.cc:1950] T 9feee73521df42ae89f5af5c5eacf9bd P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:56.143687 8356 leader_election.cc:304] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.143884 8552 tablet_replica.cc:333] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:56.144280 8552 raft_consensus.cc:2243] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.144443 9475 raft_consensus.cc:2804] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.144469 8552 raft_consensus.cc:2272] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.144516 8546 tablet_bootstrap.cc:492] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.144506 9475 raft_consensus.cc:493] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.144551 9475 raft_consensus.cc:3060] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.144586 8546 ts_tablet_manager.cc:1403] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:56.145045 8546 raft_consensus.cc:359] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.145119 8546 raft_consensus.cc:385] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.145139 8546 raft_consensus.cc:740] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.145244 8546 consensus_queue.cc:260] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.145505 8546 ts_tablet_manager.cc:1434] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.145655 8546 tablet_bootstrap.cc:492] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.145637 9475 raft_consensus.cc:515] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.146016 9475 leader_election.cc:290] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.146620 8546 tablet_bootstrap.cc:654] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.147265 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "df60914093f444fda88c13a132587513" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.147397 8284 raft_consensus.cc:3060] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.148373 8284 raft_consensus.cc:2468] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.148571 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9feee73521df42ae89f5af5c5eacf9bd (table default.Table_8cae8460736e4dcb9108757c62255907 [id=8d264830878e487daa057d286f1e3e4f]) successfully deleted
I20251028 02:59:56.148615 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "df60914093f444fda88c13a132587513" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.148963 8356 leader_election.cc:304] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.148615 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "df60914093f444fda88c13a132587513" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:56.149330 8354 leader_election.cc:343] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:56.149539 8354 leader_election.cc:343] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.149821 9475 raft_consensus.cc:2804] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.149907 9475 raft_consensus.cc:697] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:56.150044 9475 consensus_queue.cc:237] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.154448 8562 raft_consensus.cc:493] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.154552 8562 raft_consensus.cc:515] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.154953 8562 leader_election.cc:290] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.155385 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73c20437695a4043a86faa9c98091e5e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.155516 8284 raft_consensus.cc:2468] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:56.155561 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73c20437695a4043a86faa9c98091e5e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.155664 8416 raft_consensus.cc:2468] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:56.155802 8083 leader_election.cc:304] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:56.155951 8545 tablet_bootstrap.cc:492] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.155961 8562 raft_consensus.cc:2804] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.156008 8562 raft_consensus.cc:493] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.156060 8562 raft_consensus.cc:3060] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.156112 8545 ts_tablet_manager.cc:1403] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.002s
I20251028 02:59:56.156592 8545 raft_consensus.cc:359] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.156682 8545 raft_consensus.cc:385] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.156711 8545 raft_consensus.cc:740] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.156818 8545 consensus_queue.cc:260] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.157151 8545 ts_tablet_manager.cc:1434] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.157110 8562 raft_consensus.cc:515] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.157356 8545 tablet_bootstrap.cc:492] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.157511 8562 leader_election.cc:290] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.158150 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73c20437695a4043a86faa9c98091e5e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:56.158210 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73c20437695a4043a86faa9c98091e5e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.158257 8416 raft_consensus.cc:3060] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.158309 8284 raft_consensus.cc:3060] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.158438 8545 tablet_bootstrap.cc:654] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.159073 9475 raft_consensus.cc:493] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.159155 9475 raft_consensus.cc:515] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.159253 8284 raft_consensus.cc:2468] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:56.159844 9475 leader_election.cc:290] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.160041 8546 tablet_bootstrap.cc:492] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.160136 8546 ts_tablet_manager.cc:1403] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 02:59:56.160145 8544 tablet_bootstrap.cc:492] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.160614 8546 raft_consensus.cc:359] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.160710 8546 raft_consensus.cc:385] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.160739 8546 raft_consensus.cc:740] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.160863 8546 consensus_queue.cc:260] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.161170 8546 ts_tablet_manager.cc:1434] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.161212 8544 tablet_bootstrap.cc:654] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.161375 8546 tablet_bootstrap.cc:492] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.162349 8546 tablet_bootstrap.cc:654] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.159253 8416 raft_consensus.cc:2468] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:56.164987 8083 leader_election.cc:304] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:56.165411 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a8f43f58ec624ca4bb12835f2e0eecbf (table default.Table_e2689b73a8454c8dbeab470093ad1463 [id=2f8c8bd6c79d43c894e6356c6e333e7c]) successfully deleted
W20251028 02:59:56.165592 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 5155389485d54c1b82c45eb65364620f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5155389485d54c1b82c45eb65364620f already in progress: creating tablet
I20251028 02:59:56.165603 8545 tablet_bootstrap.cc:492] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.165692 8545 ts_tablet_manager.cc:1403] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
W20251028 02:59:56.165704 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet b993932fd567416984ad6d28f8cadad6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b993932fd567416984ad6d28f8cadad6 already in progress: creating tablet
I20251028 02:59:56.166195 8545 raft_consensus.cc:359] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.166299 8545 raft_consensus.cc:385] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.166330 8545 raft_consensus.cc:740] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.166424 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b93581f0eb7e499597c4e5d2f6c058aa" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.166549 8284 raft_consensus.cc:2468] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.166529 8545 consensus_queue.cc:260] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.166857 8356 leader_election.cc:304] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.167029 9366 raft_consensus.cc:2804] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.167061 9190 raft_consensus.cc:2804] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.167109 9190 raft_consensus.cc:493] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.167122 9366 raft_consensus.cc:697] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:56.167156 8545 ts_tablet_manager.cc:1434] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.167145 9190 raft_consensus.cc:3060] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.167253 9366 consensus_queue.cc:237] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.167388 8545 tablet_bootstrap.cc:492] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.168169 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b93581f0eb7e499597c4e5d2f6c058aa" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.168282 8147 raft_consensus.cc:2468] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.168357 9190 raft_consensus.cc:515] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.168547 8545 tablet_bootstrap.cc:654] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.168782 9190 leader_election.cc:290] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.169417 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b93581f0eb7e499597c4e5d2f6c058aa" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.169530 8284 raft_consensus.cc:3060] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.170509 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b93581f0eb7e499597c4e5d2f6c058aa" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.170550 8284 raft_consensus.cc:2468] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.170622 8147 raft_consensus.cc:3060] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.171180 8545 tablet_bootstrap.cc:492] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.171257 8545 ts_tablet_manager.cc:1403] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:56.171502 8356 leader_election.cc:304] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.171804 9190 raft_consensus.cc:2804] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.171895 9190 raft_consensus.cc:697] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:56.171969 8147 raft_consensus.cc:2468] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.172022 9190 consensus_queue.cc:237] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.172019 8545 raft_consensus.cc:359] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.172360 8545 raft_consensus.cc:385] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.172403 8545 raft_consensus.cc:740] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.172510 8545 consensus_queue.cc:260] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.172814 8545 ts_tablet_manager.cc:1434] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.173002 8545 tablet_bootstrap.cc:492] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.173069 8938 raft_consensus.cc:493] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.173127 8938 raft_consensus.cc:515] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.173425 8938 leader_election.cc:290] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.173784 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b5908fd54944004866ca3864e8131d2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.173908 8284 raft_consensus.cc:2468] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.174180 8356 leader_election.cc:304] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.174355 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b5908fd54944004866ca3864e8131d2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.174577 8545 tablet_bootstrap.cc:654] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.174577 8552 ts_tablet_manager.cc:1916] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.174661 8938 raft_consensus.cc:2804] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.174711 8938 raft_consensus.cc:493] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.174753 8938 raft_consensus.cc:3060] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.175475 8550 tablet_replica.cc:333] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.175622 8550 raft_consensus.cc:2243] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:56.175801 8550 pending_rounds.cc:70] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:56.175834 8550 pending_rounds.cc:77] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597140947517440 op_type: NO_OP noop_request { }
I20251028 02:59:56.175828 8938 raft_consensus.cc:515] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.175889 8550 raft_consensus.cc:2889] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:56.175940 8550 raft_consensus.cc:2272] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.176132 8938 leader_election.cc:290] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.176482 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b5908fd54944004866ca3864e8131d2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.176589 8284 raft_consensus.cc:3060] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:56.176946 8354 leader_election.cc:343] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.177313 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2b5908fd54944004866ca3864e8131d2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.177524 8284 raft_consensus.cc:2468] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
W20251028 02:59:56.177724 8354 leader_election.cc:343] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.177856 8356 leader_election.cc:304] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.178025 8938 raft_consensus.cc:2804] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.178102 8938 raft_consensus.cc:697] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:56.178234 8938 consensus_queue.cc:237] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.180541 8550 ts_tablet_manager.cc:1916] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.182224 8552 ts_tablet_manager.cc:1929] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.184221 8545 tablet_bootstrap.cc:492] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.184211 8550 ts_tablet_manager.cc:1929] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:56.184324 8545 ts_tablet_manager.cc:1403] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:56.184846 8545 raft_consensus.cc:359] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.184959 8545 raft_consensus.cc:385] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.184993 8545 raft_consensus.cc:740] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.185122 8545 consensus_queue.cc:260] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.185428 8545 ts_tablet_manager.cc:1434] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.185618 8545 tablet_bootstrap.cc:492] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.186651 8545 tablet_bootstrap.cc:654] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.183568 8546 tablet_bootstrap.cc:492] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.188544 8546 ts_tablet_manager.cc:1403] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20251028 02:59:56.189096 8546 raft_consensus.cc:359] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.189208 8546 raft_consensus.cc:385] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.189244 8546 raft_consensus.cc:740] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.189361 8546 consensus_queue.cc:260] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.189663 8546 ts_tablet_manager.cc:1434] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20251028 02:59:56.189855 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f422030a6a4042009efc7fcfb21a6bca has no consensus state
I20251028 02:59:56.189890 8546 tablet_bootstrap.cc:492] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.190089 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f2fce0c2e0b348e18d259ef0ee58e114" } new_table_name: "default.Table_5da833b34d9a44cf9f7b9ad381606ec1" modify_external_catalogs: true
I20251028 02:59:56.190924 8546 tablet_bootstrap.cc:654] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.195133 8552 log.cc:1199] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/ce64e4407ad048e8add422d77de6c211
I20251028 02:59:56.195566 8552 ts_tablet_manager.cc:1950] T ce64e4407ad048e8add422d77de6c211 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 02:59:56.196625 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2bd4558fc315491687a12efce7ee8ab2 consensus state has no leader
I20251028 02:59:56.196897 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_929b382a93794515a84b4f6356fb86b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:56.197045 8545 tablet_bootstrap.cc:492] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.197132 8545 ts_tablet_manager.cc:1403] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
W20251028 02:59:56.197220 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_929b382a93794515a84b4f6356fb86b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.197597 8545 raft_consensus.cc:359] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.197708 8545 raft_consensus.cc:385] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.197738 8545 raft_consensus.cc:740] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.197855 8545 consensus_queue.cc:260] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.198155 8545 ts_tablet_manager.cc:1434] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.198391 8545 tablet_bootstrap.cc:492] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.199472 8545 tablet_bootstrap.cc:654] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.201916 8545 tablet_bootstrap.cc:492] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.201996 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 4e44345b09584bbca19b19e57ce17c77 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.202056 8545 ts_tablet_manager.cc:1403] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:56.202356 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_892e6fce625c4e529ee8b9d81330b0d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:56.202524 8545 raft_consensus.cc:359] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.202620 8545 raft_consensus.cc:385] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.202652 8545 raft_consensus.cc:740] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.202762 8545 consensus_queue.cc:260] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.203075 8545 ts_tablet_manager.cc:1434] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.203385 8545 tablet_bootstrap.cc:492] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.203575 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet ce64e4407ad048e8add422d77de6c211 (table default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]) successfully deleted
I20251028 02:59:56.203847 8552 tablet_replica.cc:333] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:56.202661 8544 tablet_bootstrap.cc:492] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.204305 8544 ts_tablet_manager.cc:1403] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.001s
I20251028 02:59:56.204396 8545 tablet_bootstrap.cc:654] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.204878 8544 raft_consensus.cc:359] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.204981 8544 raft_consensus.cc:385] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.205013 8544 raft_consensus.cc:740] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.205139 8544 consensus_queue.cc:260] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.205420 8544 ts_tablet_manager.cc:1434] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 02:59:56.206028 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_892e6fce625c4e529ee8b9d81330b0d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.209794 8544 tablet_bootstrap.cc:492] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.216317 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2b5908fd54944004866ca3864e8131d2
I20251028 02:59:56.216420 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5bf9795bd59b42ea997a97c84e1ea111
I20251028 02:59:56.216650 8544 tablet_bootstrap.cc:654] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.216792 8562 raft_consensus.cc:493] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.216796 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_114c71e193e84c098496acdd966b7fb7" } new_table_name: "default.Table_dddd9c465f0b4e4cad048c3b984cbab3" modify_external_catalogs: true
I20251028 02:59:56.216890 8562 raft_consensus.cc:515] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.216887 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2adb2ccc4e3e4c1396032eae34eeb18c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.217270 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2adb2ccc4e3e4c1396032eae34eeb18c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.217298 8562 leader_election.cc:290] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.217880 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet 551aea93ba064bc89ae00b002eb7ccfc from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.216924 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b5908fd54944004866ca3864e8131d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.218350 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6599ec60c3004e349e7e576e4d13c4fe
I20251028 02:59:56.218614 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bf9795bd59b42ea997a97c84e1ea111 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.218782 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b5908fd54944004866ca3864e8131d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.218751 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e1f5688d9de84f4b93cb599116c876bf" } new_table_name: "default.Table_00e05a1a71944e26a7e93167c3fdce40" modify_external_catalogs: true
I20251028 02:59:56.219076 8551 tablet_replica.cc:333] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:56.219231 8551 raft_consensus.cc:2243] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:56.219439 8551 pending_rounds.cc:70] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:56.219480 8551 pending_rounds.cc:77] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597142747381760 op_type: NO_OP noop_request { }
I20251028 02:59:56.219542 8551 raft_consensus.cc:2889] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:56.219601 8551 raft_consensus.cc:2272] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.216346 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 626c2cc2d19047b794bf16c1b078941f
I20251028 02:59:56.219762 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.220057 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_8d295e2ce9374929ab925a8d8329920d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.220373 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8d295e2ce9374929ab925a8d8329920d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.220427 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bf9795bd59b42ea997a97c84e1ea111 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.221493 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 626c2cc2d19047b794bf16c1b078941f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:56.221849 8083 leader_election.cc:343] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 8a81c8bc182442bd8596251b99f90a2d
I20251028 02:59:56.222292 8551 ts_tablet_manager.cc:1916] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.222559 8550 log.cc:1199] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/19efc09c08b540a3b42c4107de5acf73
I20251028 02:59:56.222892 8550 ts_tablet_manager.cc:1950] T 19efc09c08b540a3b42c4107de5acf73 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 02:59:56.223102 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 5bf9795bd59b42ea997a97c84e1ea111 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bf9795bd59b42ea997a97c84e1ea111 already in progress: creating tablet
I20251028 02:59:56.223755 8552 raft_consensus.cc:2243] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:56.223841 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.223894 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 19efc09c08b540a3b42c4107de5acf73 (table default.Table_4b9a111a46cd448cac4abf954888e213 [id=097df645a88549b68a7c84c94f158190]) successfully deleted
I20251028 02:59:56.223965 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 626c2cc2d19047b794bf16c1b078941f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.223981 8552 pending_rounds.cc:70] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:56.218083 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bf9795bd59b42ea997a97c84e1ea111 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.224010 8552 pending_rounds.cc:77] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597141521870848 op_type: NO_OP noop_request { }
I20251028 02:59:56.218914 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b5908fd54944004866ca3864e8131d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.219586 8556 raft_consensus.cc:493] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.224081 8552 raft_consensus.cc:2889] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:56.223763 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 626c2cc2d19047b794bf16c1b078941f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.220700 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8a81c8bc182442bd8596251b99f90a2d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.224155 8556 raft_consensus.cc:515] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.223528 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.205432 8553 raft_consensus.cc:493] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.224467 8553 raft_consensus.cc:515] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.224571 8556 leader_election.cc:290] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.225013 8469 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:56.225148 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72350c60d73b41829d99ac8d01aa04bc" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.225340 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e260030c0e6423cba37a2e3af1c6400" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.225399 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e260030c0e6423cba37a2e3af1c6400" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.225450 8420 raft_consensus.cc:2468] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
W20251028 02:59:56.225692 8218 leader_election.cc:343] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.225729 8220 leader_election.cc:304] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
W20251028 02:59:56.225857 8218 leader_election.cc:343] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.225927 8556 raft_consensus.cc:2804] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.225989 8556 raft_consensus.cc:493] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.226027 8556 raft_consensus.cc:3060] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.226267 8551 ts_tablet_manager.cc:1929] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:56.226363 8551 log.cc:1199] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2b5908fd54944004866ca3864e8131d2
I20251028 02:59:56.226922 8551 ts_tablet_manager.cc:1950] T 2b5908fd54944004866ca3864e8131d2 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:56.227159 8556 raft_consensus.cc:515] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.227546 8556 leader_election.cc:290] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.228065 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72350c60d73b41829d99ac8d01aa04bc" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.228171 8416 raft_consensus.cc:2468] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:56.224833 8553 leader_election.cc:290] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.224149 8552 raft_consensus.cc:2272] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.228669 8220 leader_election.cc:304] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.228664 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e260030c0e6423cba37a2e3af1c6400" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:56.228998 8218 leader_election.cc:343] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.229219 8553 raft_consensus.cc:2804] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.229276 8553 raft_consensus.cc:493] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.229313 8553 raft_consensus.cc:3060] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.229514 8552 ts_tablet_manager.cc:1916] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.230351 8553 raft_consensus.cc:515] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.230705 8553 leader_election.cc:290] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.231067 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72350c60d73b41829d99ac8d01aa04bc" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.231066 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72350c60d73b41829d99ac8d01aa04bc" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:56.231168 8420 raft_consensus.cc:3060] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.231948 8552 ts_tablet_manager.cc:1929] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:56.232016 8552 log.cc:1199] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/8a81c8bc182442bd8596251b99f90a2d
I20251028 02:59:56.232134 8420 raft_consensus.cc:2468] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:56.232292 8552 ts_tablet_manager.cc:1950] T 8a81c8bc182442bd8596251b99f90a2d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 02:59:56.232353 8218 leader_election.cc:343] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.232514 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e260030c0e6423cba37a2e3af1c6400" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:56.232615 8416 raft_consensus.cc:3060] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.232949 8552 tablet_replica.cc:333] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:56.233086 8552 raft_consensus.cc:2243] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.233186 8552 raft_consensus.cc:2272] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.233525 8552 ts_tablet_manager.cc:1916] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.233551 8416 raft_consensus.cc:2468] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:56.234719 8220 leader_election.cc:304] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.234961 8220 leader_election.cc:304] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.235162 8083 leader_election.cc:304] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:56.235368 8562 raft_consensus.cc:2749] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.235641 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3f06af61617e4be9a753692e9ad4978d
I20251028 02:59:56.235839 8550 tablet_replica.cc:333] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.235965 7961 catalog_manager.cc:6719] ReplaceTablet: tablet 3f06af61617e4be9a753692e9ad4978d deleted and replaced by tablet 26a60624333d43f1b701036b7e9dfdb0
I20251028 02:59:56.235975 8550 raft_consensus.cc:2243] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.236086 8550 raft_consensus.cc:2272] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.236776 8546 tablet_bootstrap.cc:492] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.236861 8546 ts_tablet_manager.cc:1403] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
W20251028 02:59:56.237152 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet df60914093f444fda88c13a132587513 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df60914093f444fda88c13a132587513 already in progress: creating tablet
W20251028 02:59:56.237320 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet cbecb6472dc54989beca91ca7bcb0b57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cbecb6472dc54989beca91ca7bcb0b57 already in progress: creating tablet
I20251028 02:59:56.237351 8546 raft_consensus.cc:359] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:56.237454 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9583046f549d411a9e341560f428381c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9583046f549d411a9e341560f428381c already in progress: creating tablet
I20251028 02:59:56.237493 8546 raft_consensus.cc:385] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.237531 8546 raft_consensus.cc:740] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.237663 8546 consensus_queue.cc:260] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.237962 8546 ts_tablet_manager.cc:1434] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.237988 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 8a81c8bc182442bd8596251b99f90a2d (table default.Table_63b74aebe1364823bb6a10f4a8942db8 [id=485d972ab1864e26ace68ba52bb671ad]) successfully deleted
I20251028 02:59:56.238109 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2b5908fd54944004866ca3864e8131d2 (table default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]) successfully deleted
I20251028 02:59:56.238188 8546 tablet_bootstrap.cc:492] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:56.238214 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 6599ec60c3004e349e7e576e4d13c4fe because the tablet was not found. No further retry: Not found: Tablet not found: 6599ec60c3004e349e7e576e4d13c4fe
W20251028 02:59:56.238318 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 626c2cc2d19047b794bf16c1b078941f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 626c2cc2d19047b794bf16c1b078941f already in progress: creating tablet
I20251028 02:59:56.238675 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f06af61617e4be9a753692e9ad4978d with delete_type TABLET_DATA_DELETED (replaced by tablet 26a60624333d43f1b701036b7e9dfdb0) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.238978 8556 raft_consensus.cc:2804] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.238996 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f06af61617e4be9a753692e9ad4978d with delete_type TABLET_DATA_DELETED (replaced by tablet 26a60624333d43f1b701036b7e9dfdb0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.239059 8556 raft_consensus.cc:697] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:56.239105 9467 raft_consensus.cc:2804] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.239238 8546 tablet_bootstrap.cc:654] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.239189 8556 consensus_queue.cc:237] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.239264 9467 raft_consensus.cc:697] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:56.239391 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f06af61617e4be9a753692e9ad4978d with delete_type TABLET_DATA_DELETED (replaced by tablet 26a60624333d43f1b701036b7e9dfdb0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.239421 9467 consensus_queue.cc:237] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.240146 8333 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
W20251028 02:59:56.241314 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 3f06af61617e4be9a753692e9ad4978d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f06af61617e4be9a753692e9ad4978d already in progress: creating tablet
I20251028 02:59:56.241374 8550 ts_tablet_manager.cc:1916] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.242657 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_621dda8a15654228acd735a8dca754ef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.243245 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_621dda8a15654228acd735a8dca754ef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.243562 8544 tablet_bootstrap.cc:492] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.243665 8544 ts_tablet_manager.cc:1403] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20251028 02:59:56.245486 8546 tablet_bootstrap.cc:492] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.245579 8546 ts_tablet_manager.cc:1403] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:56.246418 8546 raft_consensus.cc:359] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.246524 8546 raft_consensus.cc:385] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.246558 8546 raft_consensus.cc:740] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.248034 8546 consensus_queue.cc:260] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.248245 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 551aea93ba064bc89ae00b002eb7ccfc
I20251028 02:59:56.248361 8546 ts_tablet_manager.cc:1434] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
W20251028 02:59:56.248414 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b9b5fb938357499b8888ba29d23612bd consensus state has no leader
I20251028 02:59:56.248548 7963 catalog_manager.cc:6719] ReplaceTablet: tablet 551aea93ba064bc89ae00b002eb7ccfc deleted and replaced by tablet a4bcdc8a0c554b7cae81d7993e4f2861
I20251028 02:59:56.248559 8546 tablet_bootstrap.cc:492] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.248656 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_552d3ae2691d42778485c7c0e3dd80bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.249073 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_552d3ae2691d42778485c7c0e3dd80bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.249253 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c90ec4690cc24b63ac21b51b090816e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.249558 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c90ec4690cc24b63ac21b51b090816e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.249688 8546 tablet_bootstrap.cc:654] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.248668 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_fa28ec70564a473fb82a04887b59f553"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.250065 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa28ec70564a473fb82a04887b59f553 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.248723 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_57d84ed216cc40969d5d75d29a6f8f5e" } new_table_name: "default.Table_90d5c8bb11484e5ab7648f8cb5d5c4e0" modify_external_catalogs: true
I20251028 02:59:56.252806 8938 consensus_queue.cc:1048] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:56.254386 8220 consensus_peers.cc:597] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a8f43f58ec624ca4bb12835f2e0eecbf. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:56.254515 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 551aea93ba064bc89ae00b002eb7ccfc with delete_type TABLET_DATA_DELETED (replaced by tablet a4bcdc8a0c554b7cae81d7993e4f2861) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.255077 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a35599c912a74f959526ab6945536e2d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:56.255280 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 551aea93ba064bc89ae00b002eb7ccfc with delete_type TABLET_DATA_DELETED (replaced by tablet a4bcdc8a0c554b7cae81d7993e4f2861) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:56.255491 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a35599c912a74f959526ab6945536e2d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.256283 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 551aea93ba064bc89ae00b002eb7ccfc with delete_type TABLET_DATA_DELETED (replaced by tablet a4bcdc8a0c554b7cae81d7993e4f2861) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.257787 8544 raft_consensus.cc:359] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.257743 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c4cf0813efcd412a97a6c42da08e8ea9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:56.257915 8544 raft_consensus.cc:385] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.257956 8544 raft_consensus.cc:740] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.258111 8544 consensus_queue.cc:260] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.259438 8544 ts_tablet_manager.cc:1434] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.016s user 0.000s sys 0.001s
I20251028 02:59:56.259634 8544 tablet_bootstrap.cc:492] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.260133 8145 raft_consensus.cc:1275] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:56.260524 8938 consensus_queue.cc:1048] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:56.261528 8544 tablet_bootstrap.cc:654] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.265931 8550 ts_tablet_manager.cc:1929] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.266657 8550 log.cc:1199] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8a81c8bc182442bd8596251b99f90a2d
W20251028 02:59:56.266836 8083 consensus_peers.cc:597] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4416e1e10f53401cbddc525d9556d107. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:56.267401 8546 tablet_bootstrap.cc:492] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.267561 8546 ts_tablet_manager.cc:1403] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20251028 02:59:56.268467 8546 raft_consensus.cc:359] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.268661 8546 raft_consensus.cc:385] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.268750 8546 raft_consensus.cc:740] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.268980 8546 consensus_queue.cc:260] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.269299 8546 ts_tablet_manager.cc:1434] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:56.269572 8546 tablet_bootstrap.cc:492] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.270387 9328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Bootstrap replayed 1/1 log segments. Stats: ops{read=369 overwritten=0 applied=363 ignored=0} inserts{seen=281 ignored=0} mutations{seen=412 ignored=0} orphaned_commits=0. Pending: 6 replicates
I20251028 02:59:56.270745 8546 tablet_bootstrap.cc:654] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.271246 9328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: Bootstrap complete.
W20251028 02:59:56.258184 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c4cf0813efcd412a97a6c42da08e8ea9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.271744 8550 ts_tablet_manager.cc:1950] T 8a81c8bc182442bd8596251b99f90a2d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.273001 8544 tablet_bootstrap.cc:492] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.273139 8544 ts_tablet_manager.cc:1403] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20251028 02:59:56.274276 8544 raft_consensus.cc:359] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.274467 8544 raft_consensus.cc:385] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.274544 8544 raft_consensus.cc:740] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
W20251028 02:59:56.274780 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 551aea93ba064bc89ae00b002eb7ccfc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 551aea93ba064bc89ae00b002eb7ccfc already in progress: creating tablet
I20251028 02:59:56.274845 8544 consensus_queue.cc:260] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.275214 8544 ts_tablet_manager.cc:1434] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:56.275744 8544 tablet_bootstrap.cc:492] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.275538 9328 raft_consensus.cc:359] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 2 FOLLOWER]: Replica starting. Triggering 6 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:56.277140 8544 tablet_bootstrap.cc:654] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.277904 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8a81c8bc182442bd8596251b99f90a2d (table default.Table_63b74aebe1364823bb6a10f4a8942db8 [id=485d972ab1864e26ace68ba52bb671ad]) successfully deleted
I20251028 02:59:56.278200 8550 tablet_replica.cc:333] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.278438 8550 raft_consensus.cc:2243] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:56.278643 8550 pending_rounds.cc:70] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:56.278728 8550 pending_rounds.cc:77] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597141335797760 op_type: NO_OP noop_request { }
I20251028 02:59:56.278874 8550 raft_consensus.cc:2889] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:56.278992 8550 raft_consensus.cc:2272] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.279577 8552 ts_tablet_manager.cc:1929] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.280910 8196 heartbeater.cc:344] Connected to a master server at 127.7.154.189:45473
I20251028 02:59:56.281539 8550 ts_tablet_manager.cc:1916] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.282565 9328 raft_consensus.cc:740] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: aac264d6d2274d3283bd3251ab77e583, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.282828 9526 tablet.cc:1722] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:56.282411 8710 mvcc.cc:204] Tried to move back new op lower bound from 7215597143066521600 to 7215597141414371328. Current Snapshot: MvccSnapshot[applied={T|T < 7215597143066521600}]
I20251028 02:59:56.283598 9328 consensus_queue.cc:260] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 363, Last appended: 2.369, Last appended by leader: 369, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 02:59:56.285416 8550 ts_tablet_manager.cc:1929] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:56.281587 8553 raft_consensus.cc:493] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.286533 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet b6d868c81a7e402ba9305bb2aa90625b (table default.Table_f99c9b345302415b87a08fe5c4777310 [id=da79674aa8cd468e84114457ee46766e]). Expected version 1 got 0
I20251028 02:59:56.286563 8553 raft_consensus.cc:515] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.286656 9328 sys_catalog.cc:565] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: configured and running, proceeding with master startup.
I20251028 02:59:56.286978 8553 leader_election.cc:290] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.283043 9467 raft_consensus.cc:493] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.287276 9467 raft_consensus.cc:515] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.287698 9467 leader_election.cc:290] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.288249 8552 log.cc:1199] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/ebc0e6d526364d4ba37e443d1e681fbc
I20251028 02:59:56.288830 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "506396e55fc242ada23d66088984f0ae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.288964 8420 raft_consensus.cc:2468] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:56.289065 8552 ts_tablet_manager.cc:1950] T ebc0e6d526364d4ba37e443d1e681fbc P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:56.289675 8220 leader_election.cc:304] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.289934 9467 raft_consensus.cc:2804] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.289988 9467 raft_consensus.cc:493] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.290026 9467 raft_consensus.cc:3060] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.290145 8546 tablet_bootstrap.cc:492] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.290230 8546 ts_tablet_manager.cc:1403] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.001s
I20251028 02:59:56.290670 8546 raft_consensus.cc:359] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.290781 8546 raft_consensus.cc:385] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.290814 8546 raft_consensus.cc:740] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.290910 8553 raft_consensus.cc:493] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.290961 8546 consensus_queue.cc:260] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.290992 8553 raft_consensus.cc:515] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.291059 9467 raft_consensus.cc:515] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.291347 9467 leader_election.cc:290] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.292466 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "506396e55fc242ada23d66088984f0ae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:56.292466 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e211a1a8c547baaee1199f7103bfad" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.292626 8420 raft_consensus.cc:3060] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.293655 8420 raft_consensus.cc:2468] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:56.293905 8546 ts_tablet_manager.cc:1434] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:56.294144 8546 tablet_bootstrap.cc:492] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.295270 8546 tablet_bootstrap.cc:654] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:56.299703 8220 leader_election.cc:343] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.299878 8220 leader_election.cc:304] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.300354 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1470187cdc74716b727052ea2b0b4a0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.300648 8546 tablet_bootstrap.cc:492] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.300742 8546 ts_tablet_manager.cc:1403] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 02:59:56.300801 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet 49b0b3459d434cf2810029ce205e745c from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.301214 8546 raft_consensus.cc:359] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.301319 8546 raft_consensus.cc:385] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.301362 8546 raft_consensus.cc:740] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
W20251028 02:59:56.301298 8220 leader_election.cc:343] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.301466 8546 consensus_queue.cc:260] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.301612 8556 raft_consensus.cc:2804] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.301687 8556 raft_consensus.cc:697] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:56.301784 8546 ts_tablet_manager.cc:1434] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.301815 8556 consensus_queue.cc:237] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.291363 8553 leader_election.cc:290] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.304189 8545 tablet_bootstrap.cc:492] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.304347 8545 ts_tablet_manager.cc:1403] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.101s user 0.002s sys 0.000s
W20251028 02:59:56.304864 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9e4eec90a2fa412bb0954805f5b8d832 consensus state has no leader
I20251028 02:59:56.304934 8545 raft_consensus.cc:359] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.305042 8545 raft_consensus.cc:385] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.305078 8545 raft_consensus.cc:740] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.305258 8545 consensus_queue.cc:260] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.306241 8545 ts_tablet_manager.cc:1434] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:56.306520 8545 tablet_bootstrap.cc:492] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.287940 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "506396e55fc242ada23d66088984f0ae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.287981 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1470187cdc74716b727052ea2b0b4a0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.291208 9529 tablet.cc:1722] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251028 02:59:56.306994 8218 leader_election.cc:343] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.291837 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e211a1a8c547baaee1199f7103bfad" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:56.307215 8218 leader_election.cc:343] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.307265 8218 leader_election.cc:304] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.291960 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "506396e55fc242ada23d66088984f0ae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:56.307684 8218 leader_election.cc:343] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.307755 8218 leader_election.cc:304] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.307868 8553 raft_consensus.cc:2749] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.307998 8553 raft_consensus.cc:2749] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.308004 9366 raft_consensus.cc:493] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251028 02:59:56.308118 8218 leader_election.cc:343] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.308112 9366 raft_consensus.cc:515] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.308468 9366 leader_election.cc:290] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.308614 8546 tablet_bootstrap.cc:492] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:56.309600 8083 leader_election.cc:343] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: ce64e4407ad048e8add422d77de6c211
I20251028 02:59:56.309787 8546 tablet_bootstrap.cc:654] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:56.310662 8083 leader_election.cc:343] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: ce64e4407ad048e8add422d77de6c211
I20251028 02:59:56.310734 8083 leader_election.cc:304] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:56.311004 9366 raft_consensus.cc:2749] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.303530 8544 tablet_bootstrap.cc:492] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.311362 8544 ts_tablet_manager.cc:1403] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20251028 02:59:56.312101 8544 raft_consensus.cc:359] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.312300 8544 raft_consensus.cc:385] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.312389 8544 raft_consensus.cc:740] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.312597 8544 consensus_queue.cc:260] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.313864 8544 ts_tablet_manager.cc:1434] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:56.314132 8544 tablet_bootstrap.cc:492] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.315764 8546 tablet_bootstrap.cc:492] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.316257 8546 ts_tablet_manager.cc:1403] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:56.317118 8546 raft_consensus.cc:359] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.317297 8546 raft_consensus.cc:385] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.317634 8546 raft_consensus.cc:740] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.317865 8546 consensus_queue.cc:260] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.318377 8546 ts_tablet_manager.cc:1434] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:56.318608 8546 tablet_bootstrap.cc:492] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.320185 8546 tablet_bootstrap.cc:654] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.321565 8544 tablet_bootstrap.cc:654] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.322515 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0ac514a2be6a4e1b904149fd8fdc8eb3" } modify_external_catalogs: true
I20251028 02:59:56.322644 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0ac514a2be6a4e1b904149fd8fdc8eb3" } modify_external_catalogs: true
I20251028 02:59:56.322671 8552 tablet_replica.cc:333] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:56.322821 8552 raft_consensus.cc:2243] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:56.323001 8552 pending_rounds.cc:70] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:56.323035 8552 pending_rounds.cc:77] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597141101535232 op_type: NO_OP noop_request { }
I20251028 02:59:56.323087 8552 raft_consensus.cc:2889] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:56.323140 8552 raft_consensus.cc:2272] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.323256 8550 log.cc:1199] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9feee73521df42ae89f5af5c5eacf9bd
I20251028 02:59:56.323591 8550 ts_tablet_manager.cc:1950] T 9feee73521df42ae89f5af5c5eacf9bd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.323830 8552 ts_tablet_manager.cc:1916] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:56.323956 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4817e3b21c474b2589d30ef9823032d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4817e3b21c474b2589d30ef9823032d2 already in progress: creating tablet
W20251028 02:59:56.324136 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 5155389485d54c1b82c45eb65364620f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5155389485d54c1b82c45eb65364620f already in progress: creating tablet
I20251028 02:59:56.325268 8545 tablet_bootstrap.cc:654] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.325874 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9feee73521df42ae89f5af5c5eacf9bd (table default.Table_8cae8460736e4dcb9108757c62255907 [id=8d264830878e487daa057d286f1e3e4f]) successfully deleted
I20251028 02:59:56.326275 8550 tablet_replica.cc:333] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.326476 8550 raft_consensus.cc:2243] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:56.326607 8552 ts_tablet_manager.cc:1929] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:56.326750 8552 log.cc:1199] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/a8f43f58ec624ca4bb12835f2e0eecbf
I20251028 02:59:56.327095 8552 ts_tablet_manager.cc:1950] T a8f43f58ec624ca4bb12835f2e0eecbf P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:56.328157 8550 pending_rounds.cc:70] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:56.328259 8550 pending_rounds.cc:77] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597141443059712 op_type: NO_OP noop_request { }
I20251028 02:59:56.328387 8550 raft_consensus.cc:2889] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:56.328507 8550 raft_consensus.cc:2272] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.329752 8550 ts_tablet_manager.cc:1916] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.330968 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet b6d868c81a7e402ba9305bb2aa90625b (table default.Table_f99c9b345302415b87a08fe5c4777310 [id=da79674aa8cd468e84114457ee46766e]). Expected version 1 got 0
I20251028 02:59:56.331172 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 2bd4558fc315491687a12efce7ee8ab2 (table default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9]). Expected version 1 got 0
I20251028 02:59:56.331267 7961 catalog_manager.cc:5649] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:56.332244 8025 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6599ec60c3004e349e7e576e4d13c4fe
I20251028 02:59:56.332966 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.333253 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet a8f43f58ec624ca4bb12835f2e0eecbf (table default.Table_e2689b73a8454c8dbeab470093ad1463 [id=2f8c8bd6c79d43c894e6356c6e333e7c]) successfully deleted
I20251028 02:59:56.333628 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.333683 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.334030 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 01d609dc2fea4c79adf0953dc98ad0af (DEFAULT_TABLE table=default.Table_5384863d743c4b8baf90bc97055948a1 [id=92b59d1a11a14497bc89b6a506956cb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.334343 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d609dc2fea4c79adf0953dc98ad0af. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:56.334384 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 6599ec60c3004e349e7e576e4d13c4fe because the tablet was not found. No further retry: Not found: Tablet not found: 6599ec60c3004e349e7e576e4d13c4fe
I20251028 02:59:56.334486 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 01d609dc2fea4c79adf0953dc98ad0af (DEFAULT_TABLE table=default.Table_5384863d743c4b8baf90bc97055948a1 [id=92b59d1a11a14497bc89b6a506956cb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.334709 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d609dc2fea4c79adf0953dc98ad0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.335062 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 01d609dc2fea4c79adf0953dc98ad0af (DEFAULT_TABLE table=default.Table_5384863d743c4b8baf90bc97055948a1 [id=92b59d1a11a14497bc89b6a506956cb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.335546 8395 tablet_service.cc:1505] Processing CreateTablet for tablet f422030a6a4042009efc7fcfb21a6bca (DEFAULT_TABLE table=default.Table_f623537faf00450f8b0997cb0e2ab81b [id=fdd6ecd598f240948882d710fd485da2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.335796 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f422030a6a4042009efc7fcfb21a6bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.335786 8128 tablet_service.cc:1505] Processing CreateTablet for tablet f422030a6a4042009efc7fcfb21a6bca (DEFAULT_TABLE table=default.Table_f623537faf00450f8b0997cb0e2ab81b [id=fdd6ecd598f240948882d710fd485da2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.336048 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f422030a6a4042009efc7fcfb21a6bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.337248 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 9a81841c22ae49c0a17931d90407accc (DEFAULT_TABLE table=default.Table_d9c99a82fc094c599c4b1b85f52b6be9 [id=06998dad5ba54b30857bdd593aaf94a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.337460 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a81841c22ae49c0a17931d90407accc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.337795 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 9a81841c22ae49c0a17931d90407accc (DEFAULT_TABLE table=default.Table_d9c99a82fc094c599c4b1b85f52b6be9 [id=06998dad5ba54b30857bdd593aaf94a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.338042 8254 tablet_service.cc:1505] Processing CreateTablet for tablet f422030a6a4042009efc7fcfb21a6bca (DEFAULT_TABLE table=default.Table_f623537faf00450f8b0997cb0e2ab81b [id=fdd6ecd598f240948882d710fd485da2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.338338 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 9a81841c22ae49c0a17931d90407accc (DEFAULT_TABLE table=default.Table_d9c99a82fc094c599c4b1b85f52b6be9 [id=06998dad5ba54b30857bdd593aaf94a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.338563 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a81841c22ae49c0a17931d90407accc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.339332 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 8665d2e933fa423d81a1b32f2ce8629b (DEFAULT_TABLE table=default.Table_5bcebe955d05492e8055eec721f46a57 [id=dc3721126b09466ab5efdb8bd4b46d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.339560 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8665d2e933fa423d81a1b32f2ce8629b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.339831 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 8665d2e933fa423d81a1b32f2ce8629b (DEFAULT_TABLE table=default.Table_5bcebe955d05492e8055eec721f46a57 [id=dc3721126b09466ab5efdb8bd4b46d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.340535 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 (DEFAULT_TABLE table=default.Table_eca57c87ce0240a0844cc210e4dbdd85 [id=289693853ee848e785d5460a21ae5063]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.340762 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.342295 8550 ts_tablet_manager.cc:1929] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:56.342464 8550 log.cc:1199] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4416e1e10f53401cbddc525d9556d107
I20251028 02:59:56.342831 8550 ts_tablet_manager.cc:1950] T 4416e1e10f53401cbddc525d9556d107 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.343065 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 8665d2e933fa423d81a1b32f2ce8629b (DEFAULT_TABLE table=default.Table_5bcebe955d05492e8055eec721f46a57 [id=dc3721126b09466ab5efdb8bd4b46d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.343351 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8665d2e933fa423d81a1b32f2ce8629b. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:56.343652 7955 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 9e4eec90a2fa412bb0954805f5b8d832 (table default.Table_90d5c8bb11484e5ab7648f8cb5d5c4e0 [id=a4f9578fb0c146a7af9769f101d57da5], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9e4eec90a2fa412bb0954805f5b8d832 consensus state has no leader
W20251028 02:59:56.343781 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet b993932fd567416984ad6d28f8cadad6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b993932fd567416984ad6d28f8cadad6 already in progress: creating tablet
I20251028 02:59:56.343935 8550 tablet_replica.cc:333] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.344100 8550 raft_consensus.cc:2243] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.344193 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4416e1e10f53401cbddc525d9556d107 (table default.Table_322bdb2bdb474e39937bd5b0ecf1556e [id=2b814ef3eb474d1fa873eaeff4fa1799]) successfully deleted
I20251028 02:59:56.344223 8550 raft_consensus.cc:2272] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:56.344344 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet aa0776d1a8ef43449309221b86ed3923 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa0776d1a8ef43449309221b86ed3923 already in progress: creating tablet
I20251028 02:59:56.344692 8550 ts_tablet_manager.cc:1916] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.344748 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a81841c22ae49c0a17931d90407accc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.344784 8552 tablet_replica.cc:333] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:56.344904 8552 raft_consensus.cc:2243] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.345001 8552 raft_consensus.cc:2272] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.345477 8552 ts_tablet_manager.cc:1916] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.346990 8550 ts_tablet_manager.cc:1929] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.347096 8550 log.cc:1199] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ce64e4407ad048e8add422d77de6c211
I20251028 02:59:56.347314 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f422030a6a4042009efc7fcfb21a6bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.347427 8550 ts_tablet_manager.cc:1950] T ce64e4407ad048e8add422d77de6c211 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.347756 8552 ts_tablet_manager.cc:1929] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.347820 8552 log.cc:1199] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/19efc09c08b540a3b42c4107de5acf73
I20251028 02:59:56.348007 8552 ts_tablet_manager.cc:1950] T 19efc09c08b540a3b42c4107de5acf73 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:56.348031 8550 tablet_replica.cc:333] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.348153 8550 raft_consensus.cc:2243] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.348273 8550 raft_consensus.cc:2272] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.348630 8550 ts_tablet_manager.cc:1916] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.348676 8552 tablet_replica.cc:333] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:56.348796 8552 raft_consensus.cc:2243] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.348878 8552 raft_consensus.cc:2272] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.349095 9551 tablet.cc:1722] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:56.349591 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d609dc2fea4c79adf0953dc98ad0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.350695 8550 ts_tablet_manager.cc:1929] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:56.350771 8550 log.cc:1199] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/0703e18d7dba4d628e3cc310be55fcf5
I20251028 02:59:56.351048 8550 ts_tablet_manager.cc:1950] T 0703e18d7dba4d628e3cc310be55fcf5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.351393 8545 tablet_bootstrap.cc:492] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.351543 8545 ts_tablet_manager.cc:1403] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.000s
I20251028 02:59:56.351658 8550 tablet_replica.cc:333] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.351819 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8665d2e933fa423d81a1b32f2ce8629b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.351822 8550 raft_consensus.cc:2243] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.351974 8550 raft_consensus.cc:2272] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.352342 8550 ts_tablet_manager.cc:1916] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.352270 8545 raft_consensus.cc:359] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.352396 8545 raft_consensus.cc:385] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.352448 8545 raft_consensus.cc:740] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.352589 8545 consensus_queue.cc:260] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.352886 8545 ts_tablet_manager.cc:1434] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.353101 8545 tablet_bootstrap.cc:492] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.354350 8545 tablet_bootstrap.cc:654] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:56.354830 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4817e3b21c474b2589d30ef9823032d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4817e3b21c474b2589d30ef9823032d2 already in progress: creating tablet
W20251028 02:59:56.355010 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet cbecb6472dc54989beca91ca7bcb0b57 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cbecb6472dc54989beca91ca7bcb0b57 already in progress: creating tablet
I20251028 02:59:56.355175 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 19efc09c08b540a3b42c4107de5acf73 (table default.Table_4b9a111a46cd448cac4abf954888e213 [id=097df645a88549b68a7c84c94f158190]) successfully deleted
W20251028 02:59:56.355283 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 9583046f549d411a9e341560f428381c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9583046f549d411a9e341560f428381c already in progress: creating tablet
I20251028 02:59:56.355921 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet ce64e4407ad048e8add422d77de6c211 (table default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]) successfully deleted
I20251028 02:59:56.356057 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 0703e18d7dba4d628e3cc310be55fcf5 (table default.Table_758c1adf3eb6460d9fa4e675a42507e3 [id=d4eaebd005bc42dd86d7125ef555546f]) successfully deleted
I20251028 02:59:56.356272 8562 raft_consensus.cc:493] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.356356 8562 raft_consensus.cc:515] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.356698 8562 leader_election.cc:290] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.357209 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb0e20b3e05c484394e11bbdea6a44ea" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.357328 8284 raft_consensus.cc:2468] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:56.357735 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb0e20b3e05c484394e11bbdea6a44ea" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.357852 8416 raft_consensus.cc:2468] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:56.358513 8083 leader_election.cc:304] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:56.358641 8550 ts_tablet_manager.cc:1929] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:56.358708 8550 log.cc:1199] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4ff8c974231f4b8aa18c18a05ad3e738
I20251028 02:59:56.358858 9401 raft_consensus.cc:2804] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.358913 9401 raft_consensus.cc:493] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.358950 9401 raft_consensus.cc:3060] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.358985 8550 ts_tablet_manager.cc:1950] T 4ff8c974231f4b8aa18c18a05ad3e738 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.359701 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]) successfully deleted
I20251028 02:59:56.359941 8550 tablet_replica.cc:333] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.360154 9401 raft_consensus.cc:515] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.360577 9401 leader_election.cc:290] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.361512 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb0e20b3e05c484394e11bbdea6a44ea" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.361635 8284 raft_consensus.cc:3060] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.362798 8284 raft_consensus.cc:2468] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:56.363142 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet b6d868c81a7e402ba9305bb2aa90625b (table default.Table_f99c9b345302415b87a08fe5c4777310 [id=da79674aa8cd468e84114457ee46766e]). Expected version 1 got 0
I20251028 02:59:56.363289 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet ac782a9f0b564667a7b20f1e328f2572 (table default.Table_dddd9c465f0b4e4cad048c3b984cbab3 [id=11719def996f48e68ac7d4ee2dbf66c5]). Expected version 1 got 0
I20251028 02:59:56.363396 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bb0e20b3e05c484394e11bbdea6a44ea" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:56.363449 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 2bd4558fc315491687a12efce7ee8ab2 (table default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9]). Expected version 1 got 0
I20251028 02:59:56.363523 8416 raft_consensus.cc:3060] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.363549 7965 catalog_manager.cc:5649] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:56.363808 7965 catalog_manager.cc:5649] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:56.364080 8553 raft_consensus.cc:493] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.364164 8553 raft_consensus.cc:515] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.364509 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a19eeac727e142579e34a301a6e2db02
I20251028 02:59:56.364559 8553 leader_election.cc:290] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.364562 8416 raft_consensus.cc:2468] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:56.364559 8938 raft_consensus.cc:493] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.364777 8938 raft_consensus.cc:515] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.365099 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b6d868c81a7e402ba9305bb2aa90625b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.365134 8938 leader_election.cc:290] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 02:59:56.365653 8218 leader_election.cc:343] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.365757 8083 leader_election.cc:304] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:56.365847 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e211a1a8c547baaee1199f7103bfad" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.366165 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b6d868c81a7e402ba9305bb2aa90625b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:56.366194 8354 leader_election.cc:343] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.366271 8416 raft_consensus.cc:2468] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:56.366328 8562 raft_consensus.cc:2804] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.366413 8562 raft_consensus.cc:697] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:56.366552 8220 leader_election.cc:304] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.366561 8562 consensus_queue.cc:237] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.366750 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b5908fd54944004866ca3864e8131d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.366792 8553 raft_consensus.cc:2804] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.366834 8553 raft_consensus.cc:493] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.366876 8553 raft_consensus.cc:3060] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.366918 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0e20b3e05c484394e11bbdea6a44ea with delete_type TABLET_DATA_DELETED (replaced by tablet bd50e7d4e28541de9799833c53fe7c02) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.367026 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 506396e55fc242ada23d66088984f0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.367139 8393 tablet_service.cc:1552] Processing DeleteTablet for tablet df60914093f444fda88c13a132587513 with delete_type TABLET_DATA_DELETED (replaced by tablet e3b33c4d189247a89878b6a8e1fcbac0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.367218 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet 929cf41ac7b0437387fbc488983f5cac with delete_type TABLET_DATA_DELETED (replaced by tablet b5e3c89222754eacb6ba4773396d62a9) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.367287 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet a19eeac727e142579e34a301a6e2db02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.367408 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 9bf28d9bd3bd462f864fc4f20468e605 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.367561 8551 tablet_replica.cc:333] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:56.367713 8551 raft_consensus.cc:2243] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.367815 8551 raft_consensus.cc:2272] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:56.367838 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 2b5908fd54944004866ca3864e8131d2 because the tablet was not found. No further retry: Not found: Tablet not found: 2b5908fd54944004866ca3864e8131d2
I20251028 02:59:56.368014 8553 raft_consensus.cc:515] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.368073 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 73c20437695a4043a86faa9c98091e5e with delete_type TABLET_DATA_DELETED (replaced by tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.368016 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet a19eeac727e142579e34a301a6e2db02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.368412 8553 leader_election.cc:290] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.369501 8551 ts_tablet_manager.cc:1916] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.369803 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 (DEFAULT_TABLE table=default.Table_eca57c87ce0240a0844cc210e4dbdd85 [id=289693853ee848e785d5460a21ae5063]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.370081 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.370289 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 (DEFAULT_TABLE table=default.Table_eca57c87ce0240a0844cc210e4dbdd85 [id=289693853ee848e785d5460a21ae5063]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.370538 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.371907 8551 ts_tablet_manager.cc:1929] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.371974 8551 log.cc:1199] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/bb0e20b3e05c484394e11bbdea6a44ea
I20251028 02:59:56.372218 8551 ts_tablet_manager.cc:1950] T bb0e20b3e05c484394e11bbdea6a44ea P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:56.372972 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b6d868c81a7e402ba9305bb2aa90625b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:56.373378 8218 leader_election.cc:343] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.374171 8551 tablet_replica.cc:333] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:56.375088 8551 raft_consensus.cc:2243] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.375241 8551 raft_consensus.cc:2272] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.375269 8550 raft_consensus.cc:2243] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.375509 8550 raft_consensus.cc:2272] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.376543 8551 ts_tablet_manager.cc:1916] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.377615 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet bb0e20b3e05c484394e11bbdea6a44ea (table default.Table_424dccd707fc4fa1bd9d01e78cab2f92 [id=dbe3cee1ce8a47fab35d8303c2950b1c]) successfully deleted
I20251028 02:59:56.378083 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 9621e42e6a324ecda50fe83c0c53119c (DEFAULT_TABLE table=default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.378337 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 6bc54972260747f3af0c18d05c6a3442 (DEFAULT_TABLE table=default.Table_861351bbb646467db2388b94e51e0d7e [id=f8375f32144c4541bea5dbcd51ab256e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.378367 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 9621e42e6a324ecda50fe83c0c53119c (DEFAULT_TABLE table=default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.374643 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b6d868c81a7e402ba9305bb2aa90625b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:56.378540 8416 raft_consensus.cc:3060] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.378583 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bc54972260747f3af0c18d05c6a3442. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.378953 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9621e42e6a324ecda50fe83c0c53119c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.379258 8551 ts_tablet_manager.cc:1929] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.379359 8551 log.cc:1199] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/506396e55fc242ada23d66088984f0ae
I20251028 02:59:56.379489 8416 raft_consensus.cc:2468] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:56.379587 8551 ts_tablet_manager.cc:1950] T 506396e55fc242ada23d66088984f0ae P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:56.380268 8551 tablet_replica.cc:333] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:56.380401 8551 raft_consensus.cc:2243] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:56.380537 8551 pending_rounds.cc:70] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:56.380509 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 506396e55fc242ada23d66088984f0ae (table default.Table_ba1740929d404081b7451d23bc910475 [id=d41b990640fd4401a2d09310961cbba4]) successfully deleted
I20251028 02:59:56.380566 8551 pending_rounds.cc:77] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597142632591360 op_type: NO_OP noop_request { }
I20251028 02:59:56.380616 8551 raft_consensus.cc:2889] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:56.380663 8551 raft_consensus.cc:2272] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.375977 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 9621e42e6a324ecda50fe83c0c53119c (DEFAULT_TABLE table=default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.376984 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 6bc54972260747f3af0c18d05c6a3442 (DEFAULT_TABLE table=default.Table_861351bbb646467db2388b94e51e0d7e [id=f8375f32144c4541bea5dbcd51ab256e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.380977 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9621e42e6a324ecda50fe83c0c53119c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.381141 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bc54972260747f3af0c18d05c6a3442. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.378429 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet a19eeac727e142579e34a301a6e2db02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.383682 8550 ts_tablet_manager.cc:1916] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.384142 8544 tablet_bootstrap.cc:492] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.384220 8544 ts_tablet_manager.cc:1403] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.070s user 0.002s sys 0.000s
I20251028 02:59:56.384716 8544 raft_consensus.cc:359] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.384838 8544 raft_consensus.cc:385] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.384876 8544 raft_consensus.cc:740] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.385001 8544 consensus_queue.cc:260] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.385298 8544 ts_tablet_manager.cc:1434] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.385493 8544 tablet_bootstrap.cc:492] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.386525 8544 tablet_bootstrap.cc:654] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.390575 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f2d0dffdf5cc414ab870e0dde75916d6
I20251028 02:59:56.390918 7968 catalog_manager.cc:6719] ReplaceTablet: tablet f2d0dffdf5cc414ab870e0dde75916d6 deleted and replaced by tablet 3b0c593a3b96434db9ac42d260fc6997
I20251028 02:59:56.391402 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet f2d0dffdf5cc414ab870e0dde75916d6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b0c593a3b96434db9ac42d260fc6997) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.392041 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet f2d0dffdf5cc414ab870e0dde75916d6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b0c593a3b96434db9ac42d260fc6997) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.393766 8550 ts_tablet_manager.cc:1929] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.393849 8550 log.cc:1199] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2b5908fd54944004866ca3864e8131d2
I20251028 02:59:56.393923 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5bcebe955d05492e8055eec721f46a57" } new_table_name: "default.Table_139255bc4fb74fae8fb86563b54826bd" modify_external_catalogs: true
I20251028 02:59:56.394068 8550 ts_tablet_manager.cc:1950] T 2b5908fd54944004866ca3864e8131d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.395244 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2b5908fd54944004866ca3864e8131d2 (table default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]) successfully deleted
W20251028 02:59:56.395426 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6599ec60c3004e349e7e576e4d13c4fe because the tablet was not found. No further retry: Not found: Tablet not found: 6599ec60c3004e349e7e576e4d13c4fe
W20251028 02:59:56.395517 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 5bf9795bd59b42ea997a97c84e1ea111 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bf9795bd59b42ea997a97c84e1ea111 already in progress: creating tablet
W20251028 02:59:56.395587 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 626c2cc2d19047b794bf16c1b078941f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 626c2cc2d19047b794bf16c1b078941f already in progress: creating tablet
W20251028 02:59:56.395663 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 3f06af61617e4be9a753692e9ad4978d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f06af61617e4be9a753692e9ad4978d already in progress: creating tablet
W20251028 02:59:56.395737 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 551aea93ba064bc89ae00b002eb7ccfc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 551aea93ba064bc89ae00b002eb7ccfc already in progress: creating tablet
W20251028 02:59:56.395850 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6599ec60c3004e349e7e576e4d13c4fe because the tablet was not found. No further retry: Not found: Tablet not found: 6599ec60c3004e349e7e576e4d13c4fe
I20251028 02:59:56.395957 8545 tablet_bootstrap.cc:492] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.396063 8545 ts_tablet_manager.cc:1403] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.000s
W20251028 02:59:56.396126 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a19eeac727e142579e34a301a6e2db02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a19eeac727e142579e34a301a6e2db02 already in progress: creating tablet
W20251028 02:59:56.396245 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f2d0dffdf5cc414ab870e0dde75916d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2d0dffdf5cc414ab870e0dde75916d6 already in progress: creating tablet
I20251028 02:59:56.396606 8545 raft_consensus.cc:359] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.396729 8545 raft_consensus.cc:385] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.396773 8545 raft_consensus.cc:740] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.396903 8545 consensus_queue.cc:260] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.378283 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e211a1a8c547baaee1199f7103bfad" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.397190 8545 ts_tablet_manager.cc:1434] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.397274 8284 raft_consensus.cc:2468] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.397378 8545 tablet_bootstrap.cc:492] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.379791 8220 leader_election.cc:304] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.391809 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet f2d0dffdf5cc414ab870e0dde75916d6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b0c593a3b96434db9ac42d260fc6997) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.397863 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9621e42e6a324ecda50fe83c0c53119c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.394492 8546 tablet_bootstrap.cc:492] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.398072 8546 ts_tablet_manager.cc:1403] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.080s user 0.002s sys 0.000s
I20251028 02:59:56.398429 8545 tablet_bootstrap.cc:654] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.398574 8546 raft_consensus.cc:359] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.398669 8546 raft_consensus.cc:385] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.398697 8546 raft_consensus.cc:740] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.398816 8546 consensus_queue.cc:260] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.399108 8546 ts_tablet_manager.cc:1434] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.399324 8546 tablet_bootstrap.cc:492] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.399467 8356 leader_election.cc:304] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.399993 8938 raft_consensus.cc:2804] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.400056 8938 raft_consensus.cc:493] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.400101 8938 raft_consensus.cc:3060] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.400295 8546 tablet_bootstrap.cc:654] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.400391 8551 ts_tablet_manager.cc:1916] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.401124 8938 raft_consensus.cc:515] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.401495 8938 leader_election.cc:290] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.401743 8965 raft_consensus.cc:2804] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.401837 8965 raft_consensus.cc:697] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:56.401983 8965 consensus_queue.cc:237] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.402652 8551 ts_tablet_manager.cc:1929] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:56.402715 8551 log.cc:1199] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/df60914093f444fda88c13a132587513
I20251028 02:59:56.402992 8551 ts_tablet_manager.cc:1950] T df60914093f444fda88c13a132587513 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:56.403513 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e211a1a8c547baaee1199f7103bfad" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.404520 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e211a1a8c547baaee1199f7103bfad" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.404619 8284 raft_consensus.cc:3060] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.405573 8284 raft_consensus.cc:2468] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.406095 8356 leader_election.cc:304] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.406275 8938 raft_consensus.cc:2804] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.406361 8938 raft_consensus.cc:697] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:56.406500 8938 consensus_queue.cc:237] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.407048 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 6bc54972260747f3af0c18d05c6a3442 (DEFAULT_TABLE table=default.Table_861351bbb646467db2388b94e51e0d7e [id=f8375f32144c4541bea5dbcd51ab256e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.407382 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bc54972260747f3af0c18d05c6a3442. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:56.407536 8354 leader_election.cc:343] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.408200 8544 tablet_bootstrap.cc:492] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.408303 8544 ts_tablet_manager.cc:1403] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20251028 02:59:56.408845 8544 raft_consensus.cc:359] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.408954 8544 raft_consensus.cc:385] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.408990 8544 raft_consensus.cc:740] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.409124 8544 consensus_queue.cc:260] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.409444 8544 ts_tablet_manager.cc:1434] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.409637 8544 tablet_bootstrap.cc:492] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.410387 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0ae27331d550469cb43063ea5db69a57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.410833 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ae27331d550469cb43063ea5db69a57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.411675 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fd8850b2aff9461ca7f69c76704fb00e" } new_table_name: "default.Table_93b5aac358764b869e8589ccc023ea0e" modify_external_catalogs: true
I20251028 02:59:56.414078 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 2ff112c66f1a4cf9b232e6bb665755f9 (DEFAULT_TABLE table=default.Table_aeee4c7fa7ae41beb0ef15ad542213e8 [id=26d281a132204bc288eda2ac08794956]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.414374 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff112c66f1a4cf9b232e6bb665755f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.416977 8544 tablet_bootstrap.cc:654] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.417541 9294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49178 (request call id 0) took 1523 ms. Trace:
I20251028 02:59:56.418063 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e40f83dab24e4e83b634a88cfe208817"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:56.418190 9284 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" instance_seqno: 1761620389689963) as {username='slave'} at 127.7.154.131:57991; Asking this server to re-register.
W20251028 02:59:56.418449 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e40f83dab24e4e83b634a88cfe208817 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.418684 9530 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:56.418784 9530 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:56.419437 9283 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" instance_seqno: 1761620389287366) as {username='slave'} at 127.7.154.129:50595; Asking this server to re-register.
I20251028 02:59:56.419797 9290 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ccbdf478f18948d19de96574a685bfda" instance_seqno: 1761620389479519) as {username='slave'} at 127.7.154.130:51593; Asking this server to re-register.
I20251028 02:59:56.420560 8196 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:56.421708 8333 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:56.422176 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_552d3ae2691d42778485c7c0e3dd80bb" } modify_external_catalogs: true
I20251028 02:59:56.422276 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_552d3ae2691d42778485c7c0e3dd80bb" } modify_external_catalogs: true
I20251028 02:59:56.422863 9286 ts_manager.cc:194] Registered new tserver with Master: fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:56.425721 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet df60914093f444fda88c13a132587513 (table default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]) successfully deleted
I20251028 02:59:56.425845 8552 ts_tablet_manager.cc:1916] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.427564 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 2ff112c66f1a4cf9b232e6bb665755f9 (DEFAULT_TABLE table=default.Table_aeee4c7fa7ae41beb0ef15ad542213e8 [id=26d281a132204bc288eda2ac08794956]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.427883 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff112c66f1a4cf9b232e6bb665755f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.428859 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 2ff112c66f1a4cf9b232e6bb665755f9 (DEFAULT_TABLE table=default.Table_aeee4c7fa7ae41beb0ef15ad542213e8 [id=26d281a132204bc288eda2ac08794956]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.429111 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ff112c66f1a4cf9b232e6bb665755f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.430948 9544 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9343ca8510de4eaa9945786e054a866d. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:56.431061 9544 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 02:59:56.431829 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_90d5c8bb11484e5ab7648f8cb5d5c4e0" } modify_external_catalogs: true
I20251028 02:59:56.431902 8469 heartbeater.cc:461] Registering TS with master...
I20251028 02:59:56.431942 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_90d5c8bb11484e5ab7648f8cb5d5c4e0" } modify_external_catalogs: true
I20251028 02:59:56.432514 9291 ts_manager.cc:194] Registered new tserver with Master: ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.432789 8551 tablet_replica.cc:333] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:56.432936 8551 raft_consensus.cc:2243] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.433044 8551 raft_consensus.cc:2272] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.434821 9291 ts_manager.cc:194] Registered new tserver with Master: d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.435319 8545 tablet_bootstrap.cc:492] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.435407 8545 ts_tablet_manager.cc:1403] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20251028 02:59:56.435913 8545 raft_consensus.cc:359] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.436028 8545 raft_consensus.cc:385] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.436065 8545 raft_consensus.cc:740] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.436187 8545 consensus_queue.cc:260] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.436472 8545 ts_tablet_manager.cc:1434] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.436673 8545 tablet_bootstrap.cc:492] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.437178 7969 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_dddd9c465f0b4e4cad048c3b984cbab3 [id=11719def996f48e68ac7d4ee2dbf66c5] alter complete (version 1)
I20251028 02:59:56.437685 8545 tablet_bootstrap.cc:654] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.445895 9541 catalog_manager.cc:1269] Loaded cluster ID: a891460fa9a9449bb0459a49a032f9be
I20251028 02:59:56.446049 8544 tablet_bootstrap.cc:492] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.446146 9541 catalog_manager.cc:1562] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: loading cluster ID for follower catalog manager: success
I20251028 02:59:56.446166 8544 ts_tablet_manager.cc:1403] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.001s
I20251028 02:59:56.446767 8544 raft_consensus.cc:359] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.446894 8544 raft_consensus.cc:385] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.446938 8544 raft_consensus.cc:740] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.447049 8544 consensus_queue.cc:260] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.447391 8544 ts_tablet_manager.cc:1434] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.449005 8544 tablet_bootstrap.cc:492] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.450412 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet b6d868c81a7e402ba9305bb2aa90625b (table default.Table_f99c9b345302415b87a08fe5c4777310 [id=da79674aa8cd468e84114457ee46766e]). Expected version 1 got 0
I20251028 02:59:56.450783 8544 tablet_bootstrap.cc:654] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.452831 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 73c20437695a4043a86faa9c98091e5e with delete_type TABLET_DATA_DELETED (replaced by tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.452960 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 9bf28d9bd3bd462f864fc4f20468e605 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.452999 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet df60914093f444fda88c13a132587513 with delete_type TABLET_DATA_DELETED (replaced by tablet e3b33c4d189247a89878b6a8e1fcbac0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.453135 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet bb0e20b3e05c484394e11bbdea6a44ea with delete_type TABLET_DATA_DELETED (replaced by tablet bd50e7d4e28541de9799833c53fe7c02) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.453193 8550 tablet_replica.cc:333] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.453325 8550 raft_consensus.cc:2243] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:56.453387 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 2b5908fd54944004866ca3864e8131d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.453482 8550 pending_rounds.cc:70] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:56.453518 8550 pending_rounds.cc:77] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597142702972928 op_type: NO_OP noop_request { }
I20251028 02:59:56.453574 8550 raft_consensus.cc:2889] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:56.453625 8550 raft_consensus.cc:2272] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.454962 7963 catalog_manager.cc:5649] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:56.455173 7963 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet b6d868c81a7e402ba9305bb2aa90625b (table default.Table_f99c9b345302415b87a08fe5c4777310 [id=da79674aa8cd468e84114457ee46766e]). Expected version 1 got 0
I20251028 02:59:56.456671 8545 tablet_bootstrap.cc:492] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.456794 8545 ts_tablet_manager.cc:1403] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251028 02:59:56.457361 8545 raft_consensus.cc:359] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.457461 8545 raft_consensus.cc:385] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.457492 8545 raft_consensus.cc:740] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.457574 8550 ts_tablet_manager.cc:1916] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.457620 8545 consensus_queue.cc:260] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.457737 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_273147e41d9f4c43b46e54c10ff21e09" } modify_external_catalogs: true
I20251028 02:59:56.457851 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_273147e41d9f4c43b46e54c10ff21e09" } modify_external_catalogs: true
I20251028 02:59:56.458231 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 35e211a1a8c547baaee1199f7103bfad with delete_type TABLET_DATA_DELETED (replaced by tablet 51198eb794764f7fbfda806fc256bcca) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.458597 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 35e211a1a8c547baaee1199f7103bfad with delete_type TABLET_DATA_DELETED (replaced by tablet 51198eb794764f7fbfda806fc256bcca) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.458668 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_46656faddd3141adbdd28b70154be5f4" } modify_external_catalogs: true
I20251028 02:59:56.458745 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 929cf41ac7b0437387fbc488983f5cac with delete_type TABLET_DATA_DELETED (replaced by tablet b5e3c89222754eacb6ba4773396d62a9) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.458755 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_46656faddd3141adbdd28b70154be5f4" } modify_external_catalogs: true
I20251028 02:59:56.460199 8550 ts_tablet_manager.cc:1929] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:56.460271 8550 log.cc:1199] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/73c20437695a4043a86faa9c98091e5e
I20251028 02:59:56.460525 8550 ts_tablet_manager.cc:1950] T 73c20437695a4043a86faa9c98091e5e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.461544 9475 raft_consensus.cc:493] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.461628 9475 raft_consensus.cc:515] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.461997 9475 leader_election.cc:290] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.462586 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a2ebe28dbd2439e963319ec6e515635" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.462718 8284 raft_consensus.cc:2468] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.462769 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a2ebe28dbd2439e963319ec6e515635" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:56.463089 8354 leader_election.cc:343] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.463189 8356 leader_election.cc:304] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.463435 9475 raft_consensus.cc:2804] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.463490 9475 raft_consensus.cc:493] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.463528 9475 raft_consensus.cc:3060] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.463662 8545 ts_tablet_manager.cc:1434] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20251028 02:59:56.463891 8545 tablet_bootstrap.cc:492] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.464563 9475 raft_consensus.cc:515] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.464958 9475 leader_election.cc:290] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.465978 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a2ebe28dbd2439e963319ec6e515635" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.466106 8284 raft_consensus.cc:3060] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.467115 8284 raft_consensus.cc:2468] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.467475 8356 leader_election.cc:304] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.467674 9475 raft_consensus.cc:2804] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.467754 9475 raft_consensus.cc:697] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:56.467890 9475 consensus_queue.cc:237] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.468081 8545 tablet_bootstrap.cc:654] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:56.469419 8220 consensus_peers.cc:597] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0703e18d7dba4d628e3cc310be55fcf5. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:56.472704 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 338cf9d02142436f97ccd44313cc8199 (DEFAULT_TABLE table=default.Table_8314d35830564f179d7e69e756fefa45 [id=5f61b3c12ed647968fc2932aa24a13f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.473035 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338cf9d02142436f97ccd44313cc8199. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.473284 8545 tablet_bootstrap.cc:492] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.473369 8545 ts_tablet_manager.cc:1403] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:56.473841 8545 raft_consensus.cc:359] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.473937 8545 raft_consensus.cc:385] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.473968 8545 raft_consensus.cc:740] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.474081 8545 consensus_queue.cc:260] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.474357 8545 ts_tablet_manager.cc:1434] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.475549 8545 tablet_bootstrap.cc:492] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.478431 8545 tablet_bootstrap.cc:654] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.480047 9475 raft_consensus.cc:493] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.480072 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 338cf9d02142436f97ccd44313cc8199 (DEFAULT_TABLE table=default.Table_8314d35830564f179d7e69e756fefa45 [id=5f61b3c12ed647968fc2932aa24a13f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.480162 9475 raft_consensus.cc:515] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.480530 9475 leader_election.cc:290] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 02:59:56.482514 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8665d2e933fa423d81a1b32f2ce8629b consensus state has no leader
I20251028 02:59:56.482709 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet f6ac7b5fff0848cebf710ba3ed50def4 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.484004 8562 raft_consensus.cc:493] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.484100 8562 raft_consensus.cc:515] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.484438 8562 leader_election.cc:290] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.484896 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e260030c0e6423cba37a2e3af1c6400" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.485033 8416 raft_consensus.cc:2393] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:56.485527 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9e260030c0e6423cba37a2e3af1c6400" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.485605 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2bd4558fc315491687a12efce7ee8ab2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.485714 8284 raft_consensus.cc:2468] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.486181 8356 leader_election.cc:304] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.486338 9475 raft_consensus.cc:2804] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.486378 9475 raft_consensus.cc:493] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.486411 9475 raft_consensus.cc:3060] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.487443 9475 raft_consensus.cc:515] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.487834 8546 tablet_bootstrap.cc:492] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.487836 9475 leader_election.cc:290] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.487929 8546 ts_tablet_manager.cc:1403] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.089s user 0.002s sys 0.000s
I20251028 02:59:56.488322 8083 leader_election.cc:304] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:56.488554 8562 raft_consensus.cc:3060] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.488512 8546 raft_consensus.cc:359] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.488633 8546 raft_consensus.cc:385] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.488667 8546 raft_consensus.cc:740] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.488803 8546 consensus_queue.cc:260] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.489068 8546 ts_tablet_manager.cc:1434] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.489255 8546 tablet_bootstrap.cc:492] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.489566 8562 raft_consensus.cc:2749] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.489841 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2bd4558fc315491687a12efce7ee8ab2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.489944 8284 raft_consensus.cc:3060] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.490341 8546 tablet_bootstrap.cc:654] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.417613 9294 rpcz_store.cc:276] 1028 02:59:54.894139 (+ 0us) service_pool.cc:168] Inserting onto call queue
1028 02:59:54.894202 (+ 63us) service_pool.cc:225] Handling call
1028 02:59:56.288806 (+1394604us) raft_consensus.cc:1471] Updating replica for 183 ops
1028 02:59:56.293592 (+ 4786us) raft_consensus.cc:1534] Early marking committed up to index 369
1028 02:59:56.293840 (+ 248us) raft_consensus.cc:1539] Triggering prepare for 183 ops
1028 02:59:56.294075 (+ 235us) write_op.cc:270] Start()
1028 02:59:56.294120 (+ 45us) write_op.cc:276] Timestamp: P: 1761620393283072 usec, L: 0
1028 02:59:56.294291 (+ 171us) write_op.cc:270] Start()
1028 02:59:56.294304 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393303732 usec, L: 0
1028 02:59:56.294396 (+ 92us) write_op.cc:270] Start()
1028 02:59:56.294408 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393304253 usec, L: 0
1028 02:59:56.294478 (+ 70us) write_op.cc:270] Start()
1028 02:59:56.294490 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393304629 usec, L: 0
1028 02:59:56.294567 (+ 77us) write_op.cc:270] Start()
1028 02:59:56.294577 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393305982 usec, L: 0
1028 02:59:56.294636 (+ 59us) write_op.cc:270] Start()
1028 02:59:56.294645 (+ 9us) write_op.cc:276] Timestamp: P: 1761620393306432 usec, L: 0
1028 02:59:56.294717 (+ 72us) write_op.cc:270] Start()
1028 02:59:56.294733 (+ 16us) write_op.cc:276] Timestamp: P: 1761620393306805 usec, L: 0
1028 02:59:56.294795 (+ 62us) write_op.cc:270] Start()
1028 02:59:56.294805 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393307122 usec, L: 0
1028 02:59:56.294866 (+ 61us) write_op.cc:270] Start()
1028 02:59:56.294876 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393307511 usec, L: 0
1028 02:59:56.294935 (+ 59us) write_op.cc:270] Start()
1028 02:59:56.294945 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393307885 usec, L: 0
1028 02:59:56.295027 (+ 82us) write_op.cc:270] Start()
1028 02:59:56.295042 (+ 15us) write_op.cc:276] Timestamp: P: 1761620393313166 usec, L: 0
1028 02:59:56.295108 (+ 66us) write_op.cc:270] Start()
1028 02:59:56.295119 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393313551 usec, L: 0
1028 02:59:56.295180 (+ 61us) write_op.cc:270] Start()
1028 02:59:56.295190 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393323743 usec, L: 0
1028 02:59:56.295246 (+ 56us) write_op.cc:270] Start()
1028 02:59:56.295256 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393325755 usec, L: 0
1028 02:59:56.295362 (+ 106us) write_op.cc:270] Start()
1028 02:59:56.295375 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393327847 usec, L: 0
1028 02:59:56.295445 (+ 70us) write_op.cc:270] Start()
1028 02:59:56.295456 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393347740 usec, L: 0
1028 02:59:56.295525 (+ 69us) write_op.cc:270] Start()
1028 02:59:56.295536 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393348249 usec, L: 0
1028 02:59:56.295599 (+ 63us) write_op.cc:270] Start()
1028 02:59:56.295610 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393375222 usec, L: 0
1028 02:59:56.295689 (+ 79us) write_op.cc:270] Start()
1028 02:59:56.295700 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393391144 usec, L: 0
1028 02:59:56.295780 (+ 80us) write_op.cc:270] Start()
1028 02:59:56.295791 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393400758 usec, L: 0
1028 02:59:56.295852 (+ 61us) write_op.cc:270] Start()
1028 02:59:56.295862 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393420885 usec, L: 0
1028 02:59:56.295928 (+ 66us) write_op.cc:270] Start()
1028 02:59:56.295938 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393425739 usec, L: 0
1028 02:59:56.296011 (+ 73us) write_op.cc:270] Start()
1028 02:59:56.296030 (+ 19us) write_op.cc:276] Timestamp: P: 1761620393426163 usec, L: 0
1028 02:59:56.296089 (+ 59us) write_op.cc:270] Start()
1028 02:59:56.296099 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393439317 usec, L: 0
1028 02:59:56.296162 (+ 63us) write_op.cc:270] Start()
1028 02:59:56.296172 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393452626 usec, L: 0
1028 02:59:56.296228 (+ 56us) write_op.cc:270] Start()
1028 02:59:56.296237 (+ 9us) write_op.cc:276] Timestamp: P: 1761620393457110 usec, L: 0
1028 02:59:56.296304 (+ 67us) write_op.cc:270] Start()
1028 02:59:56.296314 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393462582 usec, L: 0
1028 02:59:56.296373 (+ 59us) write_op.cc:270] Start()
1028 02:59:56.296383 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393463035 usec, L: 0
1028 02:59:56.296450 (+ 67us) write_op.cc:270] Start()
1028 02:59:56.296460 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393467457 usec, L: 0
1028 02:59:56.296521 (+ 61us) write_op.cc:270] Start()
1028 02:59:56.298523 (+ 2002us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x561c6c1f3330. stack: 00007f9c3fa64e43 00007f9c3fa65435 00007f9c3f5cc89d 0000561c3cb81e58 0000561c3cb81eab 0000561c3cb85df2 00007f9c43e5dfa0 00007f9c41c67bb0 00007f9c41c50b5f 00007f9c41c273a8 00007f9c41c2686d 00007f9c41752759 00007f9c41755b2a 00007f9c41751f1b 00007f9c4283bb30 00007f9c40ec7589
1028 02:59:56.298552 (+ 29us) write_op.cc:276] Timestamp: P: 1761620393473043 usec, L: 0
1028 02:59:56.298725 (+ 173us) write_op.cc:270] Start()
1028 02:59:56.298748 (+ 23us) write_op.cc:276] Timestamp: P: 1761620393503849 usec, L: 0
1028 02:59:56.298834 (+ 86us) write_op.cc:270] Start()
1028 02:59:56.298848 (+ 14us) write_op.cc:276] Timestamp: P: 1761620393516409 usec, L: 0
1028 02:59:56.298941 (+ 93us) write_op.cc:270] Start()
1028 02:59:56.298953 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393517083 usec, L: 0
1028 02:59:56.299038 (+ 85us) write_op.cc:270] Start()
1028 02:59:56.299051 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393517608 usec, L: 0
1028 02:59:56.299138 (+ 87us) write_op.cc:270] Start()
1028 02:59:56.299150 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393522310 usec, L: 0
1028 02:59:56.299220 (+ 70us) write_op.cc:270] Start()
1028 02:59:56.299232 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393523198 usec, L: 0
1028 02:59:56.299330 (+ 98us) write_op.cc:270] Start()
1028 02:59:56.299343 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393523675 usec, L: 0
1028 02:59:56.299431 (+ 88us) write_op.cc:270] Start()
1028 02:59:56.299447 (+ 16us) write_op.cc:276] Timestamp: P: 1761620393531654 usec, L: 0
1028 02:59:56.299544 (+ 97us) write_op.cc:270] Start()
1028 02:59:56.299557 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393551518 usec, L: 0
1028 02:59:56.299645 (+ 88us) write_op.cc:270] Start()
1028 02:59:56.299658 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393564633 usec, L: 0
1028 02:59:56.299731 (+ 73us) write_op.cc:270] Start()
1028 02:59:56.299742 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393566286 usec, L: 0
1028 02:59:56.299818 (+ 76us) write_op.cc:270] Start()
1028 02:59:56.299829 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393568076 usec, L: 0
1028 02:59:56.307830 (+ 8001us) write_op.cc:270] Start()
1028 02:59:56.307856 (+ 26us) write_op.cc:276] Timestamp: P: 1761620393575784 usec, L: 0
1028 02:59:56.308020 (+ 164us) write_op.cc:270] Start()
1028 02:59:56.308040 (+ 20us) write_op.cc:276] Timestamp: P: 1761620393585483 usec, L: 0
1028 02:59:56.308194 (+ 154us) write_op.cc:270] Start()
1028 02:59:56.308211 (+ 17us) write_op.cc:276] Timestamp: P: 1761620393598440 usec, L: 0
1028 02:59:56.311676 (+ 3465us) write_op.cc:270] Start()
1028 02:59:56.311708 (+ 32us) write_op.cc:276] Timestamp: P: 1761620393612853 usec, L: 0
1028 02:59:56.311818 (+ 110us) write_op.cc:270] Start()
1028 02:59:56.311832 (+ 14us) write_op.cc:276] Timestamp: P: 1761620393613341 usec, L: 0
1028 02:59:56.311904 (+ 72us) write_op.cc:270] Start()
1028 02:59:56.311916 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393625669 usec, L: 0
1028 02:59:56.312001 (+ 85us) write_op.cc:270] Start()
1028 02:59:56.312026 (+ 25us) write_op.cc:276] Timestamp: P: 1761620393635448 usec, L: 0
1028 02:59:56.312109 (+ 83us) write_op.cc:270] Start()
1028 02:59:56.312125 (+ 16us) write_op.cc:276] Timestamp: P: 1761620393640986 usec, L: 0
1028 02:59:56.312202 (+ 77us) write_op.cc:270] Start()
1028 02:59:56.312218 (+ 16us) write_op.cc:276] Timestamp: P: 1761620393641415 usec, L: 0
1028 02:59:56.312313 (+ 95us) write_op.cc:270] Start()
1028 02:59:56.312329 (+ 16us) write_op.cc:276] Timestamp: P: 1761620393648980 usec, L: 0
1028 02:59:56.312424 (+ 95us) write_op.cc:270] Start()
1028 02:59:56.312439 (+ 15us) write_op.cc:276] Timestamp: P: 1761620393650039 usec, L: 0
1028 02:59:56.312527 (+ 88us) write_op.cc:270] Start()
1028 02:59:56.312542 (+ 15us) write_op.cc:276] Timestamp: P: 1761620393654964 usec, L: 0
1028 02:59:56.312623 (+ 81us) write_op.cc:270] Start()
1028 02:59:56.312636 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393670288 usec, L: 0
1028 02:59:56.312733 (+ 97us) write_op.cc:270] Start()
1028 02:59:56.312746 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393673201 usec, L: 0
1028 02:59:56.312832 (+ 86us) write_op.cc:270] Start()
1028 02:59:56.312850 (+ 18us) write_op.cc:276] Timestamp: P: 1761620393715320 usec, L: 0
1028 02:59:56.312945 (+ 95us) write_op.cc:270] Start()
1028 02:59:56.312959 (+ 14us) write_op.cc:276] Timestamp: P: 1761620393715852 usec, L: 0
1028 02:59:56.313045 (+ 86us) write_op.cc:270] Start()
1028 02:59:56.313058 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393716307 usec, L: 0
1028 02:59:56.313143 (+ 85us) write_op.cc:270] Start()
1028 02:59:56.313154 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393716849 usec, L: 0
1028 02:59:56.313224 (+ 70us) write_op.cc:270] Start()
1028 02:59:56.313236 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393717404 usec, L: 0
1028 02:59:56.313298 (+ 62us) write_op.cc:270] Start()
1028 02:59:56.313309 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393734936 usec, L: 0
1028 02:59:56.313392 (+ 83us) write_op.cc:270] Start()
1028 02:59:56.313404 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393748343 usec, L: 0
1028 02:59:56.313477 (+ 73us) write_op.cc:270] Start()
1028 02:59:56.313488 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393760286 usec, L: 0
1028 02:59:56.313566 (+ 78us) write_op.cc:270] Start()
1028 02:59:56.313577 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393765681 usec, L: 0
1028 02:59:56.313652 (+ 75us) write_op.cc:270] Start()
1028 02:59:56.313664 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393780553 usec, L: 0
1028 02:59:56.313736 (+ 72us) write_op.cc:270] Start()
1028 02:59:56.313747 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393785209 usec, L: 0
1028 02:59:56.313830 (+ 83us) write_op.cc:270] Start()
1028 02:59:56.313843 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393785655 usec, L: 0
1028 02:59:56.313925 (+ 82us) write_op.cc:270] Start()
1028 02:59:56.313941 (+ 16us) write_op.cc:276] Timestamp: P: 1761620393786086 usec, L: 0
1028 02:59:56.314036 (+ 95us) write_op.cc:270] Start()
1028 02:59:56.314050 (+ 14us) write_op.cc:276] Timestamp: P: 1761620393786469 usec, L: 0
1028 02:59:56.314148 (+ 98us) write_op.cc:270] Start()
1028 02:59:56.314165 (+ 17us) write_op.cc:276] Timestamp: P: 1761620393787005 usec, L: 0
1028 02:59:56.314272 (+ 107us) write_op.cc:270] Start()
1028 02:59:56.314287 (+ 15us) write_op.cc:276] Timestamp: P: 1761620393787386 usec, L: 0
1028 02:59:56.314377 (+ 90us) write_op.cc:270] Start()
1028 02:59:56.314394 (+ 17us) write_op.cc:276] Timestamp: P: 1761620393787754 usec, L: 0
1028 02:59:56.314486 (+ 92us) write_op.cc:270] Start()
1028 02:59:56.314498 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393803229 usec, L: 0
1028 02:59:56.314580 (+ 82us) write_op.cc:270] Start()
1028 02:59:56.314592 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393839390 usec, L: 0
1028 02:59:56.314671 (+ 79us) write_op.cc:270] Start()
1028 02:59:56.314683 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393839927 usec, L: 0
1028 02:59:56.314750 (+ 67us) write_op.cc:270] Start()
1028 02:59:56.314761 (+ 11us) write_op.cc:276] Timestamp: P: 1761620393840905 usec, L: 0
1028 02:59:56.314832 (+ 71us) write_op.cc:270] Start()
1028 02:59:56.314858 (+ 26us) write_op.cc:276] Timestamp: P: 1761620393867026 usec, L: 0
1028 02:59:56.314944 (+ 86us) write_op.cc:270] Start()
1028 02:59:56.314957 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393884488 usec, L: 0
1028 02:59:56.318151 (+ 3194us) write_op.cc:270] Start()
1028 02:59:56.319226 (+ 1075us) write_op.cc:276] Timestamp: P: 1761620393885031 usec, L: 0
1028 02:59:56.319428 (+ 202us) write_op.cc:270] Start()
1028 02:59:56.321000 (+ 1572us) write_op.cc:276] Timestamp: P: 1761620393885446 usec, L: 0
1028 02:59:56.321162 (+ 162us) write_op.cc:270] Start()
1028 02:59:56.321178 (+ 16us) write_op.cc:276] Timestamp: P: 1761620393885858 usec, L: 0
1028 02:59:56.321255 (+ 77us) write_op.cc:270] Start()
1028 02:59:56.321267 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393904290 usec, L: 0
1028 02:59:56.321338 (+ 71us) write_op.cc:270] Start()
1028 02:59:56.321348 (+ 10us) write_op.cc:276] Timestamp: P: 1761620393915036 usec, L: 0
1028 02:59:56.321420 (+ 72us) write_op.cc:270] Start()
1028 02:59:56.321437 (+ 17us) write_op.cc:276] Timestamp: P: 1761620393928532 usec, L: 0
1028 02:59:56.321532 (+ 95us) write_op.cc:270] Start()
1028 02:59:56.322865 (+ 1333us) write_op.cc:276] Timestamp: P: 1761620393934227 usec, L: 0
1028 02:59:56.322996 (+ 131us) write_op.cc:270] Start()
1028 02:59:56.323029 (+ 33us) write_op.cc:276] Timestamp: P: 1761620393934664 usec, L: 0
1028 02:59:56.323134 (+ 105us) write_op.cc:270] Start()
1028 02:59:56.323152 (+ 18us) write_op.cc:276] Timestamp: P: 1761620393935352 usec, L: 0
1028 02:59:56.323241 (+ 89us) write_op.cc:270] Start()
1028 02:59:56.323257 (+ 16us) write_op.cc:276] Timestamp: P: 1761620393936211 usec, L: 0
1028 02:59:56.323385 (+ 128us) write_op.cc:270] Start()
1028 02:59:56.323404 (+ 19us) write_op.cc:276] Timestamp: P: 1761620393937194 usec, L: 0
1028 02:59:56.323499 (+ 95us) write_op.cc:270] Start()
1028 02:59:56.323513 (+ 14us) write_op.cc:276] Timestamp: P: 1761620393939355 usec, L: 0
1028 02:59:56.323609 (+ 96us) write_op.cc:270] Start()
1028 02:59:56.323626 (+ 17us) write_op.cc:276] Timestamp: P: 1761620393940179 usec, L: 0
1028 02:59:56.323734 (+ 108us) write_op.cc:270] Start()
1028 02:59:56.323747 (+ 13us) write_op.cc:276] Timestamp: P: 1761620393970650 usec, L: 0
1028 02:59:56.323829 (+ 82us) write_op.cc:270] Start()
1028 02:59:56.323841 (+ 12us) write_op.cc:276] Timestamp: P: 1761620393977828 usec, L: 0
1028 02:59:56.323911 (+ 70us) write_op.cc:270] Start()
1028 02:59:56.323922 (+ 11us) write_op.cc:276] Timestamp: P: 1761620394038500 usec, L: 0
1028 02:59:56.324002 (+ 80us) write_op.cc:270] Start()
1028 02:59:56.324024 (+ 22us) write_op.cc:276] Timestamp: P: 1761620394048186 usec, L: 0
1028 02:59:56.324109 (+ 85us) write_op.cc:270] Start()
1028 02:59:56.324125 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394049966 usec, L: 0
1028 02:59:56.324207 (+ 82us) write_op.cc:270] Start()
1028 02:59:56.324218 (+ 11us) write_op.cc:276] Timestamp: P: 1761620394057535 usec, L: 0
1028 02:59:56.324292 (+ 74us) write_op.cc:270] Start()
1028 02:59:56.324308 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394058448 usec, L: 0
1028 02:59:56.324418 (+ 110us) write_op.cc:270] Start()
1028 02:59:56.324434 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394058871 usec, L: 0
1028 02:59:56.324526 (+ 92us) write_op.cc:270] Start()
1028 02:59:56.324539 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394059355 usec, L: 0
1028 02:59:56.324609 (+ 70us) write_op.cc:270] Start()
1028 02:59:56.324622 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394059751 usec, L: 0
1028 02:59:56.324706 (+ 84us) write_op.cc:270] Start()
1028 02:59:56.324721 (+ 15us) write_op.cc:276] Timestamp: P: 1761620394060146 usec, L: 0
1028 02:59:56.324816 (+ 95us) write_op.cc:270] Start()
1028 02:59:56.324831 (+ 15us) write_op.cc:276] Timestamp: P: 1761620394066229 usec, L: 0
1028 02:59:56.324906 (+ 75us) write_op.cc:270] Start()
1028 02:59:56.324917 (+ 11us) write_op.cc:276] Timestamp: P: 1761620394071038 usec, L: 0
1028 02:59:56.324984 (+ 67us) write_op.cc:270] Start()
1028 02:59:56.324995 (+ 11us) write_op.cc:276] Timestamp: P: 1761620394071709 usec, L: 0
1028 02:59:56.325072 (+ 77us) write_op.cc:270] Start()
1028 02:59:56.325084 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394081412 usec, L: 0
1028 02:59:56.325184 (+ 100us) write_op.cc:270] Start()
1028 02:59:56.325202 (+ 18us) write_op.cc:276] Timestamp: P: 1761620394092829 usec, L: 0
1028 02:59:56.325293 (+ 91us) write_op.cc:270] Start()
1028 02:59:56.325310 (+ 17us) write_op.cc:276] Timestamp: P: 1761620394106291 usec, L: 0
1028 02:59:56.325393 (+ 83us) write_op.cc:270] Start()
1028 02:59:56.325405 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394111890 usec, L: 0
1028 02:59:56.325473 (+ 68us) write_op.cc:270] Start()
1028 02:59:56.325487 (+ 14us) write_op.cc:276] Timestamp: P: 1761620394133249 usec, L: 0
1028 02:59:56.325582 (+ 95us) write_op.cc:270] Start()
1028 02:59:56.325593 (+ 11us) write_op.cc:276] Timestamp: P: 1761620394135211 usec, L: 0
1028 02:59:56.325663 (+ 70us) write_op.cc:270] Start()
1028 02:59:56.325678 (+ 15us) write_op.cc:276] Timestamp: P: 1761620394136516 usec, L: 0
1028 02:59:56.325761 (+ 83us) write_op.cc:270] Start()
1028 02:59:56.325777 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394137864 usec, L: 0
1028 02:59:56.325870 (+ 93us) write_op.cc:270] Start()
1028 02:59:56.325884 (+ 14us) write_op.cc:276] Timestamp: P: 1761620394153128 usec, L: 0
1028 02:59:56.325982 (+ 98us) write_op.cc:270] Start()
1028 02:59:56.325997 (+ 15us) write_op.cc:276] Timestamp: P: 1761620394153945 usec, L: 0
1028 02:59:56.326097 (+ 100us) write_op.cc:270] Start()
1028 02:59:56.326111 (+ 14us) write_op.cc:276] Timestamp: P: 1761620394154562 usec, L: 0
1028 02:59:56.326180 (+ 69us) write_op.cc:270] Start()
1028 02:59:56.326194 (+ 14us) write_op.cc:276] Timestamp: P: 1761620394155236 usec, L: 0
1028 02:59:56.326262 (+ 68us) write_op.cc:270] Start()
1028 02:59:56.326274 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394160325 usec, L: 0
1028 02:59:56.326368 (+ 94us) write_op.cc:270] Start()
1028 02:59:56.326384 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394166315 usec, L: 0
1028 02:59:56.326455 (+ 71us) write_op.cc:270] Start()
1028 02:59:56.326466 (+ 11us) write_op.cc:276] Timestamp: P: 1761620394173181 usec, L: 0
1028 02:59:56.326544 (+ 78us) write_op.cc:270] Start()
1028 02:59:56.326558 (+ 14us) write_op.cc:276] Timestamp: P: 1761620394178764 usec, L: 0
1028 02:59:56.326629 (+ 71us) write_op.cc:270] Start()
1028 02:59:56.326640 (+ 11us) write_op.cc:276] Timestamp: P: 1761620394192598 usec, L: 0
1028 02:59:56.326734 (+ 94us) write_op.cc:270] Start()
1028 02:59:56.326746 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394195426 usec, L: 0
1028 02:59:56.326822 (+ 76us) write_op.cc:270] Start()
1028 02:59:56.326834 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394223035 usec, L: 0
1028 02:59:56.327422 (+ 588us) write_op.cc:270] Start()
1028 02:59:56.327441 (+ 19us) write_op.cc:276] Timestamp: P: 1761620394252749 usec, L: 0
1028 02:59:56.327545 (+ 104us) write_op.cc:270] Start()
1028 02:59:56.327562 (+ 17us) write_op.cc:276] Timestamp: P: 1761620394260242 usec, L: 0
1028 02:59:56.327659 (+ 97us) write_op.cc:270] Start()
1028 02:59:56.327676 (+ 17us) write_op.cc:276] Timestamp: P: 1761620394275510 usec, L: 0
1028 02:59:56.327789 (+ 113us) write_op.cc:270] Start()
1028 02:59:56.327807 (+ 18us) write_op.cc:276] Timestamp: P: 1761620394284552 usec, L: 0
1028 02:59:56.327887 (+ 80us) write_op.cc:270] Start()
1028 02:59:56.327899 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394290090 usec, L: 0
1028 02:59:56.327982 (+ 83us) write_op.cc:270] Start()
1028 02:59:56.327995 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394304212 usec, L: 0
1028 02:59:56.328097 (+ 102us) write_op.cc:270] Start()
1028 02:59:56.328110 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394304638 usec, L: 0
1028 02:59:56.328185 (+ 75us) write_op.cc:270] Start()
1028 02:59:56.328200 (+ 15us) write_op.cc:276] Timestamp: P: 1761620394304980 usec, L: 0
1028 02:59:56.328289 (+ 89us) write_op.cc:270] Start()
1028 02:59:56.328301 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394310688 usec, L: 0
1028 02:59:56.328395 (+ 94us) write_op.cc:270] Start()
1028 02:59:56.328412 (+ 17us) write_op.cc:276] Timestamp: P: 1761620394339588 usec, L: 0
1028 02:59:56.328509 (+ 97us) write_op.cc:270] Start()
1028 02:59:56.328525 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394342550 usec, L: 0
1028 02:59:56.328621 (+ 96us) write_op.cc:270] Start()
1028 02:59:56.328634 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394342985 usec, L: 0
1028 02:59:56.328719 (+ 85us) write_op.cc:270] Start()
1028 02:59:56.328732 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394343363 usec, L: 0
1028 02:59:56.328807 (+ 75us) write_op.cc:270] Start()
1028 02:59:56.328818 (+ 11us) write_op.cc:276] Timestamp: P: 1761620394343717 usec, L: 0
1028 02:59:56.328910 (+ 92us) write_op.cc:270] Start()
1028 02:59:56.328920 (+ 10us) write_op.cc:276] Timestamp: P: 1761620394344074 usec, L: 0
1028 02:59:56.328981 (+ 61us) write_op.cc:270] Start()
1028 02:59:56.328993 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394348689 usec, L: 0
1028 02:59:56.329082 (+ 89us) write_op.cc:270] Start()
1028 02:59:56.329095 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394372782 usec, L: 0
1028 02:59:56.329173 (+ 78us) write_op.cc:270] Start()
1028 02:59:56.329187 (+ 14us) write_op.cc:276] Timestamp: P: 1761620394382220 usec, L: 0
1028 02:59:56.329280 (+ 93us) write_op.cc:270] Start()
1028 02:59:56.329295 (+ 15us) write_op.cc:276] Timestamp: P: 1761620394394868 usec, L: 0
1028 02:59:56.329373 (+ 78us) write_op.cc:270] Start()
1028 02:59:56.329389 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394398506 usec, L: 0
1028 02:59:56.329469 (+ 80us) write_op.cc:270] Start()
1028 02:59:56.329481 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394403727 usec, L: 0
1028 02:59:56.329555 (+ 74us) write_op.cc:270] Start()
1028 02:59:56.329567 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394412964 usec, L: 0
1028 02:59:56.329657 (+ 90us) write_op.cc:270] Start()
1028 02:59:56.329670 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394425723 usec, L: 0
1028 02:59:56.329778 (+ 108us) write_op.cc:270] Start()
1028 02:59:56.329795 (+ 17us) write_op.cc:276] Timestamp: P: 1761620394426382 usec, L: 0
1028 02:59:56.329888 (+ 93us) write_op.cc:270] Start()
1028 02:59:56.329901 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394438961 usec, L: 0
1028 02:59:56.329968 (+ 67us) write_op.cc:270] Start()
1028 02:59:56.329981 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394446443 usec, L: 0
1028 02:59:56.347529 (+ 17548us) write_op.cc:270] Start()
1028 02:59:56.347564 (+ 35us) write_op.cc:276] Timestamp: P: 1761620394462370 usec, L: 0
1028 02:59:56.347697 (+ 133us) write_op.cc:270] Start()
1028 02:59:56.347713 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394464097 usec, L: 0
1028 02:59:56.347795 (+ 82us) write_op.cc:270] Start()
1028 02:59:56.347808 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394471124 usec, L: 0
1028 02:59:56.347879 (+ 71us) write_op.cc:270] Start()
1028 02:59:56.347892 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394479254 usec, L: 0
1028 02:59:56.347974 (+ 82us) write_op.cc:270] Start()
1028 02:59:56.347990 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394480955 usec, L: 0
1028 02:59:56.348108 (+ 118us) write_op.cc:270] Start()
1028 02:59:56.348124 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394500686 usec, L: 0
1028 02:59:56.348212 (+ 88us) write_op.cc:270] Start()
1028 02:59:56.348224 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394512080 usec, L: 0
1028 02:59:56.348310 (+ 86us) write_op.cc:270] Start()
1028 02:59:56.348323 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394513661 usec, L: 0
1028 02:59:56.348397 (+ 74us) write_op.cc:270] Start()
1028 02:59:56.348413 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394518969 usec, L: 0
1028 02:59:56.348512 (+ 99us) write_op.cc:270] Start()
1028 02:59:56.348526 (+ 14us) write_op.cc:276] Timestamp: P: 1761620394588640 usec, L: 0
1028 02:59:56.348603 (+ 77us) write_op.cc:270] Start()
1028 02:59:56.348614 (+ 11us) write_op.cc:276] Timestamp: P: 1761620394589124 usec, L: 0
1028 02:59:56.348696 (+ 82us) write_op.cc:270] Start()
1028 02:59:56.348709 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394589528 usec, L: 0
1028 02:59:56.348802 (+ 93us) write_op.cc:270] Start()
1028 02:59:56.348817 (+ 15us) write_op.cc:276] Timestamp: P: 1761620394589925 usec, L: 0
1028 02:59:56.348928 (+ 111us) write_op.cc:270] Start()
1028 02:59:56.348942 (+ 14us) write_op.cc:276] Timestamp: P: 1761620394594166 usec, L: 0
1028 02:59:56.349025 (+ 83us) write_op.cc:270] Start()
1028 02:59:56.349037 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394594714 usec, L: 0
1028 02:59:56.349105 (+ 68us) write_op.cc:270] Start()
1028 02:59:56.349147 (+ 42us) write_op.cc:276] Timestamp: P: 1761620394595128 usec, L: 0
1028 02:59:56.349246 (+ 99us) write_op.cc:270] Start()
1028 02:59:56.349258 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394595588 usec, L: 0
1028 02:59:56.349359 (+ 101us) write_op.cc:270] Start()
1028 02:59:56.349373 (+ 14us) write_op.cc:276] Timestamp: P: 1761620394623536 usec, L: 0
1028 02:59:56.349447 (+ 74us) write_op.cc:270] Start()
1028 02:59:56.349459 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394624126 usec, L: 0
1028 02:59:56.349558 (+ 99us) write_op.cc:270] Start()
1028 02:59:56.349576 (+ 18us) write_op.cc:276] Timestamp: P: 1761620394659037 usec, L: 0
1028 02:59:56.349683 (+ 107us) write_op.cc:270] Start()
1028 02:59:56.349695 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394715569 usec, L: 0
1028 02:59:56.349814 (+ 119us) write_op.cc:270] Start()
1028 02:59:56.349829 (+ 15us) write_op.cc:276] Timestamp: P: 1761620394716399 usec, L: 0
1028 02:59:56.349919 (+ 90us) write_op.cc:270] Start()
1028 02:59:56.349932 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394717024 usec, L: 0
1028 02:59:56.350015 (+ 83us) write_op.cc:270] Start()
1028 02:59:56.350028 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394717540 usec, L: 0
1028 02:59:56.350104 (+ 76us) write_op.cc:270] Start()
1028 02:59:56.350116 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394720350 usec, L: 0
1028 02:59:56.350201 (+ 85us) write_op.cc:270] Start()
1028 02:59:56.350214 (+ 13us) write_op.cc:276] Timestamp: P: 1761620394721780 usec, L: 0
1028 02:59:56.350298 (+ 84us) write_op.cc:270] Start()
1028 02:59:56.350310 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394722661 usec, L: 0
1028 02:59:56.350391 (+ 81us) write_op.cc:270] Start()
1028 02:59:56.350403 (+ 12us) write_op.cc:276] Timestamp: P: 1761620394723434 usec, L: 0
1028 02:59:56.350497 (+ 94us) write_op.cc:270] Start()
1028 02:59:56.350513 (+ 16us) write_op.cc:276] Timestamp: P: 1761620394751158 usec, L: 0
1028 02:59:56.350625 (+ 112us) write_op.cc:270] Start()
1028 02:59:56.350642 (+ 17us) write_op.cc:276] Timestamp: P: 1761620394759780 usec, L: 0
1028 02:59:56.355155 (+ 4513us) write_op.cc:270] Start()
1028 02:59:56.355184 (+ 29us) write_op.cc:276] Timestamp: P: 1761620394760536 usec, L: 0
1028 02:59:56.355323 (+ 139us) write_op.cc:270] Start()
1028 02:59:56.355342 (+ 19us) write_op.cc:276] Timestamp: P: 1761620394761187 usec, L: 0
1028 02:59:56.359960 (+ 4618us) log.cc:844] Serialized 112900 byte log entry
1028 02:59:56.360148 (+ 188us) raft_consensus.cc:1645] Marking committed up to 551
1028 02:59:56.360717 (+ 569us) write_op.cc:489] Releasing partition, row and schema locks
1028 02:59:56.360720 (+ 3us) write_op.cc:454] Released schema lock
1028 02:59:56.361153 (+ 433us) write_op.cc:489] Releasing partition, row and schema locks
1028 02:59:56.361155 (+ 2us) write_op.cc:454] Released schema lock
1028 02:59:56.361540 (+ 385us) write_op.cc:489] Releasing partition, row and schema locks
1028 02:59:56.361543 (+ 3us) write_op.cc:454] Released schema lock
1028 02:59:56.361908 (+ 365us) write_op.cc:489] Releasing partition, row and schema locks
1028 02:59:56.361911 (+ 3us) write_op.cc:454] Released schema lock
1028 02:59:56.362280 (+ 369us) write_op.cc:489] Releasing partition, row and schema locks
1028 02:59:56.362284 (+ 4us) write_op.cc:454] Released schema lock
1028 02:59:56.362610 (+ 326us) write_op.cc:489] Releasing partition, row and schema locks
1028 02:59:56.362615 (+ 5us) write_op.cc:454] Released schema lock
1028 02:59:56.362921 (+ 306us) write_op.cc:489] Releasing partition, row and schema locks
1028 02:59:56.362926 (+ 5us) write_op.cc:454] Released schema lock
1028 02:59:56.363282 (+ 356us) write_op.cc:489] Releasing partition, row and schema locks
1028 02:59:56.363284 (+ 2us) write_op.cc:454] Released schema lock
1028 02:59:56.363622 (+ 338us) write_op.cc:489] Releasing partition, row and schema locks
1028 02:59:56.363626 (+ 4us) write_op.cc:454] Released schema lock
1028 02:59:56.363953 (+ 327us) write_op.cc:489] Releasing partition
I20251028 02:59:56.491053 8284 raft_consensus.cc:2468] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.491365 8356 leader_election.cc:304] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.491634 9475 raft_consensus.cc:2804] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.491712 9475 raft_consensus.cc:697] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:56.491847 9475 consensus_queue.cc:237] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.494164 9475 raft_consensus.cc:493] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.494258 9475 raft_consensus.cc:515] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.494607 9475 leader_election.cc:290] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.495879 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "369397a89baa472285b25acd6c78e7ec" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.496071 8284 raft_consensus.cc:2468] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.496353 8356 leader_election.cc:304] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.498136 9475 raft_consensus.cc:2804] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.498198 9475 raft_consensus.cc:493] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.498234 9475 raft_consensus.cc:3060] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.499253 9475 raft_consensus.cc:515] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.499432 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a2ebe28dbd2439e963319ec6e515635" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.499487 9417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:56.499596 9417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: This master's current role is: LEADER
I20251028 02:59:56.499734 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2bd4558fc315491687a12efce7ee8ab2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.499436 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2bd4558fc315491687a12efce7ee8ab2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.500070 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "369397a89baa472285b25acd6c78e7ec" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:56.500166 8354 leader_election.cc:343] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:56.500319 8354 leader_election.cc:343] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.500381 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338cf9d02142436f97ccd44313cc8199. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:56.500459 8354 leader_election.cc:343] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:56.500705 8354 leader_election.cc:343] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.501253 9475 leader_election.cc:290] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.501457 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "369397a89baa472285b25acd6c78e7ec" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.501582 8284 raft_consensus.cc:3060] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.502653 9517 raft_consensus.cc:493] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.502756 9517 raft_consensus.cc:515] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.502784 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "369397a89baa472285b25acd6c78e7ec" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:56.503116 8354 leader_election.cc:343] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.503216 9517 leader_election.cc:290] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.503412 8284 raft_consensus.cc:2468] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.503643 8356 leader_election.cc:304] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.503907 9475 raft_consensus.cc:2804] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
W20251028 02:59:56.503984 8220 leader_election.cc:343] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:56.504067 9475 raft_consensus.cc:697] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:56.504215 9475 consensus_queue.cc:237] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:56.504539 8218 leader_election.cc:343] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:56.504603 8218 leader_election.cc:304] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.504809 9517 raft_consensus.cc:2749] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.505337 8550 tablet_replica.cc:333] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.505328 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 73c20437695a4043a86faa9c98091e5e (table default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
I20251028 02:59:56.505554 8550 raft_consensus.cc:2243] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.505684 8550 raft_consensus.cc:2272] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.506253 8550 ts_tablet_manager.cc:1916] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.508437 8551 ts_tablet_manager.cc:1916] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.509068 8544 tablet_bootstrap.cc:492] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.509161 8544 ts_tablet_manager.cc:1403] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.060s user 0.000s sys 0.002s
I20251028 02:59:56.509361 8254 tablet_service.cc:1505] Processing CreateTablet for tablet 338cf9d02142436f97ccd44313cc8199 (DEFAULT_TABLE table=default.Table_8314d35830564f179d7e69e756fefa45 [id=5f61b3c12ed647968fc2932aa24a13f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.509625 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338cf9d02142436f97ccd44313cc8199. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.509658 8544 raft_consensus.cc:359] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.509776 8544 raft_consensus.cc:385] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.509807 8544 raft_consensus.cc:740] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.509925 8544 consensus_queue.cc:260] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.510216 8544 ts_tablet_manager.cc:1434] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:56.510403 8544 tablet_bootstrap.cc:492] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.514427 8550 ts_tablet_manager.cc:1929] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.514587 8550 log.cc:1199] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9bf28d9bd3bd462f864fc4f20468e605
I20251028 02:59:56.514647 8546 tablet_bootstrap.cc:492] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.514732 8546 ts_tablet_manager.cc:1403] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.000s
I20251028 02:59:56.515051 8550 ts_tablet_manager.cc:1950] T 9bf28d9bd3bd462f864fc4f20468e605 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.515714 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9bf28d9bd3bd462f864fc4f20468e605 (table default.Table_ce36e2a0a4374085b1e87288b3288ab9 [id=e61fa366db844911bb80af844de41fa6]) successfully deleted
I20251028 02:59:56.515820 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 12d103e9b5024a87aa232c26e0c51ab9 (DEFAULT_TABLE table=default.Table_12b6b08740be470382334a74e5df73cb [id=1b3819ae7b6f4269a5be015632d5c786]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.515874 8546 raft_consensus.cc:359] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.515981 8546 raft_consensus.cc:385] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.516034 8546 raft_consensus.cc:740] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.516069 9517 raft_consensus.cc:493] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.516122 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12d103e9b5024a87aa232c26e0c51ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.516151 9517 raft_consensus.cc:515] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.516194 8546 consensus_queue.cc:260] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.516554 9517 leader_election.cc:290] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.517086 9475 raft_consensus.cc:493] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.517177 9475 raft_consensus.cc:515] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.517328 8546 ts_tablet_manager.cc:1434] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251028 02:59:56.517489 8546 tablet_bootstrap.cc:492] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.517861 9475 leader_election.cc:290] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.517988 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06302d2cb5554805b85916741ea10f0c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.518006 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a344e2e03e4eeaa688275e9ecbf63f" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.518101 8284 raft_consensus.cc:2468] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
W20251028 02:59:56.518324 8220 leader_election.cc:343] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.518322 8356 leader_election.cc:304] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.518426 8546 tablet_bootstrap.cc:654] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.518507 9475 raft_consensus.cc:2804] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.518563 9475 raft_consensus.cc:493] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.518601 9475 raft_consensus.cc:3060] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.519447 8550 tablet_replica.cc:333] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.519601 9475 raft_consensus.cc:515] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.519649 8550 raft_consensus.cc:2243] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.519819 8550 raft_consensus.cc:2272] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.520437 8550 ts_tablet_manager.cc:1916] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.521734 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06302d2cb5554805b85916741ea10f0c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.521807 9475 leader_election.cc:290] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.522002 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a344e2e03e4eeaa688275e9ecbf63f" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.522050 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a344e2e03e4eeaa688275e9ecbf63f" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.522149 8284 raft_consensus.cc:3060] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.522192 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a344e2e03e4eeaa688275e9ecbf63f" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.522360 8545 tablet_bootstrap.cc:492] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.522452 8545 ts_tablet_manager.cc:1403] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
W20251028 02:59:56.522505 8354 leader_election.cc:343] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:56.522662 8218 leader_election.cc:343] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.522732 8218 leader_election.cc:304] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:56.522738 8354 leader_election.cc:343] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.522929 9517 raft_consensus.cc:2749] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.522954 8545 raft_consensus.cc:359] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.523056 8545 raft_consensus.cc:385] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.523098 8545 raft_consensus.cc:740] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.523214 8545 consensus_queue.cc:260] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.523547 8545 ts_tablet_manager.cc:1434] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.523747 8545 tablet_bootstrap.cc:492] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.523751 8284 raft_consensus.cc:2468] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.524011 8356 leader_election.cc:304] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.524277 8938 raft_consensus.cc:2804] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.524350 8938 raft_consensus.cc:697] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:56.524474 8938 consensus_queue.cc:237] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.524703 8545 tablet_bootstrap.cc:654] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.525094 8544 tablet_bootstrap.cc:654] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.525682 8546 tablet_bootstrap.cc:492] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.525832 8546 ts_tablet_manager.cc:1403] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20251028 02:59:56.527099 8546 raft_consensus.cc:359] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.527266 8546 raft_consensus.cc:385] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.527403 8546 raft_consensus.cc:740] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.527570 8546 consensus_queue.cc:260] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.529170 8546 ts_tablet_manager.cc:1434] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:56.529402 8546 tablet_bootstrap.cc:492] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.531651 8546 tablet_bootstrap.cc:654] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.534816 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 12d103e9b5024a87aa232c26e0c51ab9 (DEFAULT_TABLE table=default.Table_12b6b08740be470382334a74e5df73cb [id=1b3819ae7b6f4269a5be015632d5c786]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.535778 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 12d103e9b5024a87aa232c26e0c51ab9 (DEFAULT_TABLE table=default.Table_12b6b08740be470382334a74e5df73cb [id=1b3819ae7b6f4269a5be015632d5c786]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.536089 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12d103e9b5024a87aa232c26e0c51ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.536098 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12d103e9b5024a87aa232c26e0c51ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.499274 9541 catalog_manager.cc:1584] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: acquiring CA information for follower catalog manager: success
I20251028 02:59:56.539697 8550 ts_tablet_manager.cc:1929] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.539880 8550 log.cc:1199] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/df60914093f444fda88c13a132587513
I20251028 02:59:56.540488 8550 ts_tablet_manager.cc:1950] T df60914093f444fda88c13a132587513 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.541714 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet df60914093f444fda88c13a132587513 (table default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]) successfully deleted
I20251028 02:59:56.543414 8550 tablet_replica.cc:333] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.543540 8551 ts_tablet_manager.cc:1929] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251028 02:59:56.539526 8218 consensus_peers.cc:597] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda -> Peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Couldn't send request to peer fbd2afb4894c4e7a8520cc09f8593aa0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ff8c974231f4b8aa18c18a05ad3e738. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:56.543917 9517 raft_consensus.cc:493] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.544013 9517 raft_consensus.cc:515] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.544466 9517 leader_election.cc:290] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.544823 8552 ts_tablet_manager.cc:1929] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.545069 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "929cf41ac7b0437387fbc488983f5cac" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.545661 8550 raft_consensus.cc:2243] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:56.545800 8544 tablet_bootstrap.cc:492] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.545914 8544 ts_tablet_manager.cc:1403] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20251028 02:59:56.545945 8550 pending_rounds.cc:70] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:56.546069 8550 pending_rounds.cc:77] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597143518830592 op_type: NO_OP noop_request { }
I20251028 02:59:56.546198 8550 raft_consensus.cc:2889] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:56.546423 8544 raft_consensus.cc:359] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.548208 8544 raft_consensus.cc:385] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.548261 8544 raft_consensus.cc:740] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.548396 8544 consensus_queue.cc:260] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.548698 8544 ts_tablet_manager.cc:1434] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:56.548938 8544 tablet_bootstrap.cc:492] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 02:59:56.549777 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e9a28ff8c0ee46cb89ba7e28de2cce45 consensus state has no leader
I20251028 02:59:56.550083 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d9c99a82fc094c599c4b1b85f52b6be9" } new_table_name: "default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7" modify_external_catalogs: true
I20251028 02:59:56.550611 8544 tablet_bootstrap.cc:654] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.551190 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a044870c006d4bde90c803732430cbe5" } new_table_name: "default.Table_594bc3babcbf48e4b5d9490f18936563" modify_external_catalogs: true
I20251028 02:59:56.552790 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_cc716ae9a4cd4b939b2e461342f03331"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.553158 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc716ae9a4cd4b939b2e461342f03331 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.554215 8550 raft_consensus.cc:2272] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.554971 8550 ts_tablet_manager.cc:1916] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.557814 8125 tablet_service.cc:1505] Processing CreateTablet for tablet e45fe8888fbc415ea637b363dccdd06b (DEFAULT_TABLE table=default.Table_1562d90171b44a268e4777c51b3c8483 [id=7dc1def3a92c46cd8995a3b46c4e2231]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:56.545519 8218 leader_election.cc:343] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.558121 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e45fe8888fbc415ea637b363dccdd06b. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:56.549139 8220 consensus_peers.cc:597] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ff8c974231f4b8aa18c18a05ad3e738. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:56.555430 8546 tablet_bootstrap.cc:492] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.558750 8546 ts_tablet_manager.cc:1403] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 02:59:56.559691 8546 raft_consensus.cc:359] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.559854 8546 raft_consensus.cc:385] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.559912 8546 raft_consensus.cc:740] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.560077 8546 consensus_queue.cc:260] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.560376 8546 ts_tablet_manager.cc:1434] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:56.560719 8552 log.cc:1199] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/df60914093f444fda88c13a132587513
I20251028 02:59:56.561021 8552 ts_tablet_manager.cc:1950] T df60914093f444fda88c13a132587513 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:56.561691 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet df60914093f444fda88c13a132587513 (table default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]) successfully deleted
I20251028 02:59:56.561849 8546 tablet_bootstrap.cc:492] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.563097 8546 tablet_bootstrap.cc:654] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.564865 8550 ts_tablet_manager.cc:1929] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:56.566886 7962 catalog_manager.cc:5931] Not sending DeleteTablet requests; no consensus state for tablet 3ac908a9551f4fc09dd812d157772d56
I20251028 02:59:56.567443 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9e4eec90a2fa412bb0954805f5b8d832
I20251028 02:59:56.567835 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f170e997649f4ff3a51b21a5ee733768"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.568223 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f170e997649f4ff3a51b21a5ee733768 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.568910 9517 raft_consensus.cc:493] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.569000 9517 raft_consensus.cc:515] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.569396 9517 leader_election.cc:290] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.570202 8550 log.cc:1199] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/bb0e20b3e05c484394e11bbdea6a44ea
I20251028 02:59:56.570524 8550 ts_tablet_manager.cc:1950] T bb0e20b3e05c484394e11bbdea6a44ea P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.571691 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet bb0e20b3e05c484394e11bbdea6a44ea (table default.Table_424dccd707fc4fa1bd9d01e78cab2f92 [id=dbe3cee1ce8a47fab35d8303c2950b1c]) successfully deleted
I20251028 02:59:56.571767 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49b0b3459d434cf2810029ce205e745c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 02:59:56.571903 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 2b5908fd54944004866ca3864e8131d2 because the tablet was not found. No further retry: Not found: Tablet not found: 2b5908fd54944004866ca3864e8131d2
W20251028 02:59:56.572211 8218 leader_election.cc:343] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.567121 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 5c1994484abf4b6fbbc3f47527558cdb (DEFAULT_TABLE table=default.Table_a74b1be50de04891b89cb1445a48a40e [id=b0407c8f78a84245a954c67e888da3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.573659 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c1994484abf4b6fbbc3f47527558cdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.574133 9541 catalog_manager.cc:1612] T 00000000000000000000000000000000 P aac264d6d2274d3283bd3251ab77e583: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251028 02:59:56.575333 8544 tablet_bootstrap.cc:492] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.575418 8544 ts_tablet_manager.cc:1403] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.003s
I20251028 02:59:56.575457 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e4eec90a2fa412bb0954805f5b8d832 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.575937 8544 raft_consensus.cc:359] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.570078 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e4eec90a2fa412bb0954805f5b8d832 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.576043 8544 raft_consensus.cc:385] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.576104 8544 raft_consensus.cc:740] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.576238 8544 consensus_queue.cc:260] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.576507 8544 ts_tablet_manager.cc:1434] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.576486 8254 tablet_service.cc:1505] Processing CreateTablet for tablet 5c1994484abf4b6fbbc3f47527558cdb (DEFAULT_TABLE table=default.Table_a74b1be50de04891b89cb1445a48a40e [id=b0407c8f78a84245a954c67e888da3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.576704 8544 tablet_bootstrap.cc:492] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.576746 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c1994484abf4b6fbbc3f47527558cdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.577697 8544 tablet_bootstrap.cc:654] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:56.580255 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9e4eec90a2fa412bb0954805f5b8d832 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9e4eec90a2fa412bb0954805f5b8d832 already in progress: creating tablet
I20251028 02:59:56.580461 9517 raft_consensus.cc:493] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.580539 9517 raft_consensus.cc:515] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:56.580776 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet aa0776d1a8ef43449309221b86ed3923 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa0776d1a8ef43449309221b86ed3923 already in progress: creating tablet
I20251028 02:59:56.580922 9517 leader_election.cc:290] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.580973 8552 tablet_replica.cc:333] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:56.581117 8552 raft_consensus.cc:2243] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.581218 8552 raft_consensus.cc:2272] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.581595 8552 ts_tablet_manager.cc:1916] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.581760 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13de247dfb18478daac9ca5601220faa" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.544869 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "929cf41ac7b0437387fbc488983f5cac" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.545361 8545 tablet_bootstrap.cc:492] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.582340 8545 ts_tablet_manager.cc:1403] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.000s
I20251028 02:59:56.582441 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe (DEFAULT_TABLE table=default.Table_4f4721dca71f432b82e320f8fa9b0cf5 [id=bc3f005bc9424552831e2ecca5d049a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.582715 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6599ec60c3004e349e7e576e4d13c4fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.582860 8545 raft_consensus.cc:359] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.582962 8545 raft_consensus.cc:385] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.582989 8545 raft_consensus.cc:740] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.583104 8545 consensus_queue.cc:260] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.586476 8545 ts_tablet_manager.cc:1434] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:56.586694 8545 tablet_bootstrap.cc:492] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.587934 8545 tablet_bootstrap.cc:654] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.588758 8938 consensus_queue.cc:1048] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:56.590389 8544 tablet_bootstrap.cc:492] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.590487 8544 ts_tablet_manager.cc:1403] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20251028 02:59:56.590955 8544 raft_consensus.cc:359] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.591080 8544 raft_consensus.cc:385] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.591111 8544 raft_consensus.cc:740] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.547055 9475 raft_consensus.cc:493] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.591235 8544 consensus_queue.cc:260] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.591233 9475 raft_consensus.cc:515] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.591569 8544 ts_tablet_manager.cc:1434] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.591776 8544 tablet_bootstrap.cc:492] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.591940 9475 leader_election.cc:290] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 02:59:56.592269 8356 leader_election.cc:343] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: 9bf28d9bd3bd462f864fc4f20468e605
I20251028 02:59:56.593377 8544 tablet_bootstrap.cc:654] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.562105 8399 tablet_service.cc:1505] Processing CreateTablet for tablet e45fe8888fbc415ea637b363dccdd06b (DEFAULT_TABLE table=default.Table_1562d90171b44a268e4777c51b3c8483 [id=7dc1def3a92c46cd8995a3b46c4e2231]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.562739 9190 raft_consensus.cc:493] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.594606 9190 raft_consensus.cc:515] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.595010 9190 leader_election.cc:290] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.571444 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49b0b3459d434cf2810029ce205e745c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:56.595466 8354 leader_election.cc:343] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 9bf28d9bd3bd462f864fc4f20468e605
I20251028 02:59:56.595530 8354 leader_election.cc:304] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.595721 9190 raft_consensus.cc:2749] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.596450 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e45fe8888fbc415ea637b363dccdd06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.574795 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e4eec90a2fa412bb0954805f5b8d832 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.596562 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7f6a7ada824eb1bca9be5344c9287e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.580431 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 5c1994484abf4b6fbbc3f47527558cdb (DEFAULT_TABLE table=default.Table_a74b1be50de04891b89cb1445a48a40e [id=b0407c8f78a84245a954c67e888da3ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.596690 8147 raft_consensus.cc:2468] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.596916 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c1994484abf4b6fbbc3f47527558cdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.598335 8354 leader_election.cc:304] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:56.598493 9190 raft_consensus.cc:2804] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.598546 9190 raft_consensus.cc:493] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.598582 9190 raft_consensus.cc:3060] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.581357 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13de247dfb18478daac9ca5601220faa" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.599524 9190 raft_consensus.cc:515] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.599895 9190 leader_election.cc:290] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.601164 8551 log.cc:1199] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/929cf41ac7b0437387fbc488983f5cac
I20251028 02:59:56.601491 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7f6a7ada824eb1bca9be5344c9287e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.601624 8147 raft_consensus.cc:3060] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.602660 8147 raft_consensus.cc:2468] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.603590 8354 leader_election.cc:304] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:56.605291 9190 raft_consensus.cc:2804] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.605386 9190 raft_consensus.cc:697] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:56.605526 9190 consensus_queue.cc:237] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.609290 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet c03faadb312742a4aecac0325f3595b7 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.612181 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 506396e55fc242ada23d66088984f0ae with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.612401 8550 tablet_replica.cc:333] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.612572 8550 raft_consensus.cc:2243] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.612699 8550 raft_consensus.cc:2272] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.613047 8544 tablet_bootstrap.cc:492] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.613150 8544 ts_tablet_manager.cc:1403] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20251028 02:59:56.613217 8550 ts_tablet_manager.cc:1916] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.613680 8544 raft_consensus.cc:359] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.613791 8544 raft_consensus.cc:385] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.613826 8544 raft_consensus.cc:740] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.613974 8544 consensus_queue.cc:260] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.614303 8544 ts_tablet_manager.cc:1434] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.614513 8544 tablet_bootstrap.cc:492] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.615799 8550 ts_tablet_manager.cc:1929] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.615866 8550 log.cc:1199] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/506396e55fc242ada23d66088984f0ae
I20251028 02:59:56.616070 8550 ts_tablet_manager.cc:1950] T 506396e55fc242ada23d66088984f0ae P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.616734 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 506396e55fc242ada23d66088984f0ae (table default.Table_ba1740929d404081b7451d23bc910475 [id=d41b990640fd4401a2d09310961cbba4]) successfully deleted
I20251028 02:59:56.619948 8544 tablet_bootstrap.cc:654] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.624884 8544 tablet_bootstrap.cc:492] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.625011 8544 ts_tablet_manager.cc:1403] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:56.625795 8544 raft_consensus.cc:359] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.625934 8544 raft_consensus.cc:385] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.625973 8544 raft_consensus.cc:740] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.626101 8544 consensus_queue.cc:260] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.626418 8544 ts_tablet_manager.cc:1434] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.630155 8544 tablet_bootstrap.cc:492] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.630599 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe (DEFAULT_TABLE table=default.Table_4f4721dca71f432b82e320f8fa9b0cf5 [id=bc3f005bc9424552831e2ecca5d049a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.630895 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6599ec60c3004e349e7e576e4d13c4fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.631480 8116 tablet_service.cc:1505] Processing CreateTablet for tablet fd8c04efd8c2456b9c7643641c7fbc0d (DEFAULT_TABLE table=default.Table_327424a7e56243ae97d58c0f6fd560fd [id=009632a3749c4bbe9cf4e856758b9a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.631713 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8c04efd8c2456b9c7643641c7fbc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.633478 8395 tablet_service.cc:1505] Processing CreateTablet for tablet fd8c04efd8c2456b9c7643641c7fbc0d (DEFAULT_TABLE table=default.Table_327424a7e56243ae97d58c0f6fd560fd [id=009632a3749c4bbe9cf4e856758b9a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.633718 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8c04efd8c2456b9c7643641c7fbc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.634042 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 4c0081e0ce4d4e499c37f217fe62c546 (DEFAULT_TABLE table=default.Table_e9eca9a92e57463bbf1e10276cd7e50a [id=35a4d845b3fd47259c8d1db7901b2e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.634253 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0081e0ce4d4e499c37f217fe62c546. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.634443 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f657c3fd0e15427e97d689cdc7266b04"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.634830 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f657c3fd0e15427e97d689cdc7266b04 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.636003 8394 tablet_service.cc:1505] Processing CreateTablet for tablet 4c0081e0ce4d4e499c37f217fe62c546 (DEFAULT_TABLE table=default.Table_e9eca9a92e57463bbf1e10276cd7e50a [id=35a4d845b3fd47259c8d1db7901b2e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.636253 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0081e0ce4d4e499c37f217fe62c546. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.637389 7969 catalog_manager.cc:5649] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:56.637681 7969 catalog_manager.cc:5649] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:56.637861 7969 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 06302d2cb5554805b85916741ea10f0c (table default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d]). Expected version 2 got 0
I20251028 02:59:56.637954 7969 catalog_manager.cc:5649] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:56.638139 7969 catalog_manager.cc:5649] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:56.638263 7969 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 2bd4558fc315491687a12efce7ee8ab2 (table default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9]). Expected version 1 got 0
I20251028 02:59:56.639168 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_87426eb930ad44e5902eb3aee3da0d71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.639609 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_87426eb930ad44e5902eb3aee3da0d71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.641798 8544 tablet_bootstrap.cc:654] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.641006 9475 consensus_queue.cc:1048] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:56.633719 8551 ts_tablet_manager.cc:1950] T 929cf41ac7b0437387fbc488983f5cac P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:56.643484 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 929cf41ac7b0437387fbc488983f5cac (table default.Table_aa25a85ac8d24940a727a586d6c630db [id=10f40fa1652f495e91fcbd5852bc3464]) successfully deleted
I20251028 02:59:56.647859 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a76e16d10a174adda26b8ddd49ee528c
I20251028 02:59:56.649643 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet a76e16d10a174adda26b8ddd49ee528c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:56.650148 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a76e16d10a174adda26b8ddd49ee528c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a76e16d10a174adda26b8ddd49ee528c already in progress: creating tablet
I20251028 02:59:56.651401 8394 tablet_service.cc:1552] Processing DeleteTablet for tablet a76e16d10a174adda26b8ddd49ee528c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:56.582183 8218 leader_election.cc:343] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:56.652146 8218 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.154.129:46529, user_credentials={real_user=slave}} blocked reactor thread for 69969.3us
W20251028 02:59:56.585706 8220 leader_election.cc:343] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: Tablet not RUNNING: STOPPED
I20251028 02:59:56.652633 8220 leader_election.cc:304] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:56.652805 8220 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.154.131:33189, user_credentials={real_user=slave}} blocked reactor thread for 67109.7us
W20251028 02:59:56.653072 8220 leader_election.cc:343] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.653126 8220 leader_election.cc:304] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:56.653381 8220 leader_election.cc:343] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.653434 8220 leader_election.cc:304] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.653923 8553 raft_consensus.cc:493] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.653997 8553 raft_consensus.cc:515] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.654333 8553 leader_election.cc:290] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.654410 8553 raft_consensus.cc:2749] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.654744 8553 raft_consensus.cc:2749] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.654840 8553 raft_consensus.cc:2749] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.588326 8546 tablet_bootstrap.cc:492] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.596156 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7f6a7ada824eb1bca9be5344c9287e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.603149 8281 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7f6a7ada824eb1bca9be5344c9287e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.654997 8546 ts_tablet_manager.cc:1403] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.093s user 0.002s sys 0.000s
I20251028 02:59:56.655021 8281 raft_consensus.cc:3060] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.655520 8546 raft_consensus.cc:359] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.655625 8546 raft_consensus.cc:385] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.655655 8546 raft_consensus.cc:740] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.655753 8546 consensus_queue.cc:260] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.655812 8544 tablet_bootstrap.cc:492] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.655905 8544 ts_tablet_manager.cc:1403] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.001s
I20251028 02:59:56.656059 8546 ts_tablet_manager.cc:1434] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:56.656246 8546 tablet_bootstrap.cc:492] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.656286 8281 raft_consensus.cc:2468] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.617003 9517 raft_consensus.cc:493] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.656483 9517 raft_consensus.cc:515] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.656463 8544 raft_consensus.cc:359] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.656567 8544 raft_consensus.cc:385] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.656598 8544 raft_consensus.cc:740] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.656735 8544 consensus_queue.cc:260] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.656823 9517 leader_election.cc:290] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.657034 8544 ts_tablet_manager.cc:1434] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.657169 8284 raft_consensus.cc:2376] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1. Re-sending same reply.
I20251028 02:59:56.657856 8546 tablet_bootstrap.cc:654] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.661834 8544 tablet_bootstrap.cc:492] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.662915 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fef84b69b1b746d4b4feeed96670c67b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.663053 8544 tablet_bootstrap.cc:654] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.663210 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7f6a7ada824eb1bca9be5344c9287e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.663363 8147 raft_consensus.cc:2393] T bf7f6a7ada824eb1bca9be5344c9287e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
W20251028 02:59:56.663857 8218 leader_election.cc:343] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.664197 9467 consensus_queue.cc:1048] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:56.664734 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bf7f6a7ada824eb1bca9be5344c9287e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.664878 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fef84b69b1b746d4b4feeed96670c67b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.664996 8419 raft_consensus.cc:2468] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:56.665844 8546 tablet_bootstrap.cc:492] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.666007 8546 ts_tablet_manager.cc:1403] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.003s
I20251028 02:59:56.666582 8546 raft_consensus.cc:359] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.666764 8546 raft_consensus.cc:385] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.666877 8546 raft_consensus.cc:740] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.667162 8546 consensus_queue.cc:260] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.629715 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe (DEFAULT_TABLE table=default.Table_4f4721dca71f432b82e320f8fa9b0cf5 [id=bc3f005bc9424552831e2ecca5d049a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.667651 8546 ts_tablet_manager.cc:1434] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:56.667840 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6599ec60c3004e349e7e576e4d13c4fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.668890 8546 tablet_bootstrap.cc:492] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.670102 8546 tablet_bootstrap.cc:654] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.631664 8258 tablet_service.cc:1505] Processing CreateTablet for tablet fd8c04efd8c2456b9c7643641c7fbc0d (DEFAULT_TABLE table=default.Table_327424a7e56243ae97d58c0f6fd560fd [id=009632a3749c4bbe9cf4e856758b9a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.670614 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8c04efd8c2456b9c7643641c7fbc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.633178 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 4c0081e0ce4d4e499c37f217fe62c546 (DEFAULT_TABLE table=default.Table_e9eca9a92e57463bbf1e10276cd7e50a [id=35a4d845b3fd47259c8d1db7901b2e8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.672897 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0081e0ce4d4e499c37f217fe62c546. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.676483 8544 tablet_bootstrap.cc:492] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.619877 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 26a344e2e03e4eeaa688275e9ecbf63f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.651909 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet a76e16d10a174adda26b8ddd49ee528c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.667230 8220 leader_election.cc:304] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.676604 8544 ts_tablet_manager.cc:1403] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 02:59:56.676867 8220 leader_election.cc:304] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.677114 9467 raft_consensus.cc:2749] T bf7f6a7ada824eb1bca9be5344c9287e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.677120 8544 raft_consensus.cc:359] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.677225 8544 raft_consensus.cc:385] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.677255 8544 raft_consensus.cc:740] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.677345 9467 raft_consensus.cc:2804] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.677387 9467 raft_consensus.cc:493] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.677377 8544 consensus_queue.cc:260] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.677424 9467 raft_consensus.cc:3060] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.677641 8544 ts_tablet_manager.cc:1434] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.678325 9467 raft_consensus.cc:515] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.678682 9467 leader_election.cc:290] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.679478 8544 tablet_bootstrap.cc:492] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.680713 8544 tablet_bootstrap.cc:654] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.681504 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fef84b69b1b746d4b4feeed96670c67b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:56.682076 8218 leader_election.cc:343] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.682662 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fef84b69b1b746d4b4feeed96670c67b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:56.682791 8416 raft_consensus.cc:3060] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.683924 8416 raft_consensus.cc:2468] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:56.684474 8220 leader_election.cc:304] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:56.684878 9467 raft_consensus.cc:2804] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.684969 9467 raft_consensus.cc:697] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:56.685109 9467 consensus_queue.cc:237] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.686116 8556 consensus_queue.cc:1048] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:56.686409 9190 raft_consensus.cc:493] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.686503 9190 raft_consensus.cc:515] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.690347 8562 raft_consensus.cc:493] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.690448 8562 raft_consensus.cc:515] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.690881 8562 leader_election.cc:290] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.691193 8544 tablet_bootstrap.cc:492] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.691290 8544 ts_tablet_manager.cc:1403] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20251028 02:59:56.691731 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "551aea93ba064bc89ae00b002eb7ccfc" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.691769 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "551aea93ba064bc89ae00b002eb7ccfc" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.691867 8416 raft_consensus.cc:2468] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:56.691857 8544 raft_consensus.cc:359] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.691912 8284 raft_consensus.cc:2468] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:56.691951 8544 raft_consensus.cc:385] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.691980 8544 raft_consensus.cc:740] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.692111 8544 consensus_queue.cc:260] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.692200 8083 leader_election.cc:304] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:56.692394 9401 raft_consensus.cc:2804] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.692421 8544 ts_tablet_manager.cc:1434] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.692440 9401 raft_consensus.cc:493] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.692481 9401 raft_consensus.cc:3060] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.692601 8544 tablet_bootstrap.cc:492] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.693746 8544 tablet_bootstrap.cc:654] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.694180 9401 raft_consensus.cc:515] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.694552 9401 leader_election.cc:290] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.696497 8281 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "551aea93ba064bc89ae00b002eb7ccfc" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.696625 8281 raft_consensus.cc:3060] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.697701 8281 raft_consensus.cc:2468] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:56.698020 8083 leader_election.cc:304] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:56.698200 8562 raft_consensus.cc:2804] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.698263 8562 raft_consensus.cc:697] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:56.698382 8562 consensus_queue.cc:237] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.698784 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "551aea93ba064bc89ae00b002eb7ccfc" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:56.698905 8416 raft_consensus.cc:3060] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.699967 8416 raft_consensus.cc:2468] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:56.702441 9190 leader_election.cc:290] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.703572 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1470187cdc74716b727052ea2b0b4a0" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.703694 8145 raft_consensus.cc:2468] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.704146 8281 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1470187cdc74716b727052ea2b0b4a0" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.704279 8281 raft_consensus.cc:2468] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.704679 8356 leader_election.cc:304] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.704972 8938 raft_consensus.cc:2804] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.705052 8938 raft_consensus.cc:493] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.705099 8938 raft_consensus.cc:3060] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.705840 8544 tablet_bootstrap.cc:492] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.706013 8544 ts_tablet_manager.cc:1403] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.001s
I20251028 02:59:56.706125 8938 raft_consensus.cc:515] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.706506 8938 leader_election.cc:290] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.706689 8544 raft_consensus.cc:359] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.706787 8544 raft_consensus.cc:385] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.706820 8544 raft_consensus.cc:740] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.706944 8544 consensus_queue.cc:260] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.707445 8544 ts_tablet_manager.cc:1434] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.707691 8544 tablet_bootstrap.cc:492] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.707679 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1470187cdc74716b727052ea2b0b4a0" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.707840 8284 raft_consensus.cc:3060] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.708902 8544 tablet_bootstrap.cc:654] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.708884 8284 raft_consensus.cc:2468] T c1470187cdc74716b727052ea2b0b4a0 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.709214 8356 leader_election.cc:304] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.709692 8938 raft_consensus.cc:2804] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.709803 8938 raft_consensus.cc:697] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:56.709941 8938 consensus_queue.cc:237] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.710330 8544 tablet_bootstrap.cc:492] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.710408 8544 ts_tablet_manager.cc:1403] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:56.710906 8544 raft_consensus.cc:359] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.711004 8544 raft_consensus.cc:385] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.711036 8544 raft_consensus.cc:740] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.711155 8544 consensus_queue.cc:260] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.711472 8544 ts_tablet_manager.cc:1434] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.711654 8544 tablet_bootstrap.cc:492] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.712880 8544 tablet_bootstrap.cc:654] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.714802 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c1470187cdc74716b727052ea2b0b4a0" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.714926 8145 raft_consensus.cc:3060] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.716073 8145 raft_consensus.cc:2468] T c1470187cdc74716b727052ea2b0b4a0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.723845 8416 raft_consensus.cc:1275] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:56.724440 9467 consensus_queue.cc:1048] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:56.724781 8147 raft_consensus.cc:3060] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.726015 8147 raft_consensus.cc:1275] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251028 02:59:56.726383 8218 consensus_peers.cc:597] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda -> Peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Couldn't send request to peer fbd2afb4894c4e7a8520cc09f8593aa0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 506396e55fc242ada23d66088984f0ae. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:56.727428 8553 consensus_queue.cc:1048] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:56.731545 9551 tablet.cc:1722] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:56.739128 9526 tablet.cc:1722] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:56.740507 8544 tablet_bootstrap.cc:492] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.740630 8544 ts_tablet_manager.cc:1403] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 02:59:56.741204 8544 raft_consensus.cc:359] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.741330 8544 raft_consensus.cc:385] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.741374 8544 raft_consensus.cc:740] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.741509 8544 consensus_queue.cc:260] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.744488 9190 raft_consensus.cc:493] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.744588 9190 raft_consensus.cc:515] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.745218 9190 leader_election.cc:290] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.745414 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06302d2cb5554805b85916741ea10f0c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.745453 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06302d2cb5554805b85916741ea10f0c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.745553 8147 raft_consensus.cc:2468] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.745579 8284 raft_consensus.cc:2468] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:56.745857 8356 leader_election.cc:304] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.746096 9190 raft_consensus.cc:2804] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.746192 9190 raft_consensus.cc:493] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.746282 9190 raft_consensus.cc:3060] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.747430 9190 raft_consensus.cc:515] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.748142 9190 leader_election.cc:290] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.748885 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06302d2cb5554805b85916741ea10f0c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:56.749007 8284 raft_consensus.cc:3060] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.749213 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06302d2cb5554805b85916741ea10f0c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.749326 8147 raft_consensus.cc:3060] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.750237 8284 raft_consensus.cc:2468] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.750424 8147 raft_consensus.cc:2468] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:56.750635 8356 leader_election.cc:304] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.751364 8553 consensus_queue.cc:1048] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:56.751538 8544 ts_tablet_manager.cc:1434] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:56.752002 8544 tablet_bootstrap.cc:492] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.752024 9190 raft_consensus.cc:2804] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.752177 9190 raft_consensus.cc:697] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:56.752377 9190 consensus_queue.cc:237] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.753181 8544 tablet_bootstrap.cc:654] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.760638 9529 tablet.cc:1722] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:56.763686 8544 tablet_bootstrap.cc:492] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.763897 8544 ts_tablet_manager.cc:1403] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:56.764441 8544 raft_consensus.cc:359] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.764633 8544 raft_consensus.cc:385] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.764724 8544 raft_consensus.cc:740] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.764945 8544 consensus_queue.cc:260] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.643936 8257 tablet_service.cc:1505] Processing CreateTablet for tablet e45fe8888fbc415ea637b363dccdd06b (DEFAULT_TABLE table=default.Table_1562d90171b44a268e4777c51b3c8483 [id=7dc1def3a92c46cd8995a3b46c4e2231]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.768392 8544 ts_tablet_manager.cc:1434] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:56.768416 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e45fe8888fbc415ea637b363dccdd06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.771497 8544 tablet_bootstrap.cc:492] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.772676 8544 tablet_bootstrap.cc:654] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:56.773226 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a19eeac727e142579e34a301a6e2db02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a19eeac727e142579e34a301a6e2db02 already in progress: creating tablet
I20251028 02:59:56.775203 8145 raft_consensus.cc:3060] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.777264 9596 consensus_queue.cc:1048] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:56.782770 8551 tablet_replica.cc:333] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:56.782948 8551 raft_consensus.cc:2243] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.783116 8551 raft_consensus.cc:2272] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.783515 8551 ts_tablet_manager.cc:1916] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.785962 8551 ts_tablet_manager.cc:1929] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.787001 8544 tablet_bootstrap.cc:492] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.787110 8544 ts_tablet_manager.cc:1403] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20251028 02:59:56.787600 8544 raft_consensus.cc:359] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.787714 8544 raft_consensus.cc:385] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.787748 8544 raft_consensus.cc:740] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.787875 8544 consensus_queue.cc:260] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.788138 8544 ts_tablet_manager.cc:1434] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.792963 8544 tablet_bootstrap.cc:492] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.794430 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1c41d49f9eba40a48bd6900671297db9
I20251028 02:59:56.795599 8544 tablet_bootstrap.cc:654] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.795857 8253 tablet_service.cc:1505] Processing CreateTablet for tablet b7d3062426434c339be4a745decae729 (DEFAULT_TABLE table=default.Table_4045ef6d27fd4f2fbfb8975411994305 [id=1dd28e8d2834401589250978261308c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.796170 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7d3062426434c339be4a745decae729. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.796254 8145 raft_consensus.cc:3060] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.799118 8125 tablet_service.cc:1505] Processing CreateTablet for tablet b7d3062426434c339be4a745decae729 (DEFAULT_TABLE table=default.Table_4045ef6d27fd4f2fbfb8975411994305 [id=1dd28e8d2834401589250978261308c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.799258 8128 tablet_service.cc:1505] Processing CreateTablet for tablet ad926e9ee0044c62a7abbfca959a4f56 (DEFAULT_TABLE table=default.Table_6ad31a79d9f043628bdac1dd2d4c263c [id=4570d7ddf33c447fb40d6411ea47e52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.799424 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7d3062426434c339be4a745decae729. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.799607 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad926e9ee0044c62a7abbfca959a4f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.799765 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c41d49f9eba40a48bd6900671297db9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.799923 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c41d49f9eba40a48bd6900671297db9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.800029 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 1c41d49f9eba40a48bd6900671297db9 (DEFAULT_TABLE table=default.Table_46656faddd3141adbdd28b70154be5f4 [id=ab17c384ca9242f094e9373db2ef9c7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.800217 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c41d49f9eba40a48bd6900671297db9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.800194 8255 tablet_service.cc:1505] Processing CreateTablet for tablet ad926e9ee0044c62a7abbfca959a4f56 (DEFAULT_TABLE table=default.Table_6ad31a79d9f043628bdac1dd2d4c263c [id=4570d7ddf33c447fb40d6411ea47e52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.800407 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad926e9ee0044c62a7abbfca959a4f56. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:56.803018 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1c41d49f9eba40a48bd6900671297db9 because the tablet was not found. No further retry: Not found: Tablet not found: 1c41d49f9eba40a48bd6900671297db9
I20251028 02:59:56.804270 8126 tablet_service.cc:1505] Processing CreateTablet for tablet e3b33c4d189247a89878b6a8e1fcbac0 (DEFAULT_TABLE table=default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.804507 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3b33c4d189247a89878b6a8e1fcbac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.804849 8254 tablet_service.cc:1505] Processing CreateTablet for tablet 1c41d49f9eba40a48bd6900671297db9 (DEFAULT_TABLE table=default.Table_46656faddd3141adbdd28b70154be5f4 [id=ab17c384ca9242f094e9373db2ef9c7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.805092 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c41d49f9eba40a48bd6900671297db9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.806026 8544 tablet_bootstrap.cc:492] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.806118 8544 ts_tablet_manager.cc:1403] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20251028 02:59:56.806613 8544 raft_consensus.cc:359] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.806720 8544 raft_consensus.cc:385] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.806752 8544 raft_consensus.cc:740] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.806864 8544 consensus_queue.cc:260] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.807163 8544 ts_tablet_manager.cc:1434] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.808183 8125 tablet_service.cc:1505] Processing CreateTablet for tablet f39ed19223fa4268a01aaf4c7762d86e (DEFAULT_TABLE table=default.Table_4debdeef26b14d9b93a878210eaba4c3 [id=5a391450a3a04f01bcd5c3c06d36cf66]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.809814 8126 tablet_service.cc:1505] Processing CreateTablet for tablet ba64524142934af08658e434e7fb08ed (DEFAULT_TABLE table=default.Table_8b7624342986474fbd0e31c86947f781 [id=530470c553154d76bf0df08ddfbf2fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.810230 8544 tablet_bootstrap.cc:492] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.811722 8253 tablet_service.cc:1505] Processing CreateTablet for tablet ba64524142934af08658e434e7fb08ed (DEFAULT_TABLE table=default.Table_8b7624342986474fbd0e31c86947f781 [id=530470c553154d76bf0df08ddfbf2fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.812011 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba64524142934af08658e434e7fb08ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.812539 8544 tablet_bootstrap.cc:654] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.813638 8544 tablet_bootstrap.cc:492] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.813714 8544 ts_tablet_manager.cc:1403] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:56.814141 8544 raft_consensus.cc:359] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.814236 8544 raft_consensus.cc:385] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.814263 8544 raft_consensus.cc:740] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.814288 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f39ed19223fa4268a01aaf4c7762d86e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.814374 8544 consensus_queue.cc:260] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.814642 8544 ts_tablet_manager.cc:1434] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.816277 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 8dab70ca2d9f4b5fba338cb337b0b396 (DEFAULT_TABLE table=default.Table_f4e5df9294de4390bfbaaf9e86ade7c7 [id=eab0e9b296d04340b7a2fd250f68d9f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.816532 8544 tablet_bootstrap.cc:492] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.817431 8544 tablet_bootstrap.cc:654] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.817934 8255 tablet_service.cc:1505] Processing CreateTablet for tablet e3b33c4d189247a89878b6a8e1fcbac0 (DEFAULT_TABLE table=default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.818217 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3b33c4d189247a89878b6a8e1fcbac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.819103 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba64524142934af08658e434e7fb08ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.819739 8254 tablet_service.cc:1505] Processing CreateTablet for tablet 8dab70ca2d9f4b5fba338cb337b0b396 (DEFAULT_TABLE table=default.Table_f4e5df9294de4390bfbaaf9e86ade7c7 [id=eab0e9b296d04340b7a2fd250f68d9f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.819949 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dab70ca2d9f4b5fba338cb337b0b396. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.822860 8544 tablet_bootstrap.cc:492] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.822949 8544 ts_tablet_manager.cc:1403] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20251028 02:59:56.823395 8544 raft_consensus.cc:359] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.823489 8544 raft_consensus.cc:385] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.823515 8544 raft_consensus.cc:740] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.823622 8544 consensus_queue.cc:260] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.823835 8544 ts_tablet_manager.cc:1434] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.824012 8544 tablet_bootstrap.cc:492] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.824997 8544 tablet_bootstrap.cc:654] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:56.826272 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9a81841c22ae49c0a17931d90407accc consensus state has no leader
I20251028 02:59:56.826576 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0325897e2eb5414685dd8be9e6d2fd2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.827003 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0325897e2eb5414685dd8be9e6d2fd2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.827804 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f6ac7b5fff0848cebf710ba3ed50def4
I20251028 02:59:56.828092 7968 catalog_manager.cc:6719] ReplaceTablet: tablet f6ac7b5fff0848cebf710ba3ed50def4 deleted and replaced by tablet 4f90e3c3524e4498aa6c4bfc5304d44b
I20251028 02:59:56.828230 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 45572222af854b34a5f26a00f8ad9615 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.828747 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet f6ac7b5fff0848cebf710ba3ed50def4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f90e3c3524e4498aa6c4bfc5304d44b) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:56.829798 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f6ac7b5fff0848cebf710ba3ed50def4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6ac7b5fff0848cebf710ba3ed50def4 already in progress: creating tablet
I20251028 02:59:56.829911 8544 tablet_bootstrap.cc:492] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.829998 8544 ts_tablet_manager.cc:1403] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:56.830441 8544 raft_consensus.cc:359] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.830542 8544 raft_consensus.cc:385] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.830581 8544 raft_consensus.cc:740] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.830713 8544 consensus_queue.cc:260] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.831020 8544 ts_tablet_manager.cc:1434] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.831314 8544 tablet_bootstrap.cc:492] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.831404 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5a485f2c39204b9b8ed300bb7e924bed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.831775 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a485f2c39204b9b8ed300bb7e924bed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.832338 8544 tablet_bootstrap.cc:654] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.832698 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_78c54cae3ad1472a993dc8ba15399ea2" } new_table_name: "default.Table_71dd113f5d12423d9aca617205523aa1" modify_external_catalogs: true
I20251028 02:59:56.833642 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6ebfbcbecf7c4975ac638a06859f32d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.833978 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ebfbcbecf7c4975ac638a06859f32d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:56.836176 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c0f428e5c6fe43fc8f78761d868ddd09 consensus state has no leader
I20251028 02:59:56.836400 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_35aa9cb3afdc4a149341fa51b840258f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.836716 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35aa9cb3afdc4a149341fa51b840258f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.837777 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet a9653fc3236f4e0b812de5a859932587 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.838481 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet f6ac7b5fff0848cebf710ba3ed50def4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f90e3c3524e4498aa6c4bfc5304d44b) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.842204 8544 tablet_bootstrap.cc:492] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.842389 8544 ts_tablet_manager.cc:1403] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:56.843061 8544 raft_consensus.cc:359] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.843168 8544 raft_consensus.cc:385] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.843204 8544 raft_consensus.cc:740] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.843402 8544 consensus_queue.cc:260] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.844497 8544 ts_tablet_manager.cc:1434] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 02:59:56.846585 8544 tablet_bootstrap.cc:492] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 02:59:56.847518 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 consensus state has no leader
I20251028 02:59:56.847785 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1" } modify_external_catalogs: true
I20251028 02:59:56.847895 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1" } modify_external_catalogs: true
I20251028 02:59:56.848924 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 45572222af854b34a5f26a00f8ad9615
I20251028 02:59:56.849224 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 45572222af854b34a5f26a00f8ad9615 deleted and replaced by tablet d60fc654f6af44b290d46235082a4113
I20251028 02:59:56.849352 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 9621e42e6a324ecda50fe83c0c53119c from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.849382 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 45572222af854b34a5f26a00f8ad9615 with delete_type TABLET_DATA_DELETED (replaced by tablet d60fc654f6af44b290d46235082a4113) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.850284 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 45572222af854b34a5f26a00f8ad9615 with delete_type TABLET_DATA_DELETED (replaced by tablet d60fc654f6af44b290d46235082a4113) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.850562 8544 tablet_bootstrap.cc:654] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:56.850551 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 45572222af854b34a5f26a00f8ad9615 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45572222af854b34a5f26a00f8ad9615 already in progress: creating tablet
I20251028 02:59:56.850739 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_663247205d844151bccc25218dcf39a5" } new_table_name: "default.Table_66f629fcdad8471bb69ea0c30537b139" modify_external_catalogs: true
I20251028 02:59:56.851655 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_8c6b8c4cb0e84fb88487023453581cec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.852020 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c6b8c4cb0e84fb88487023453581cec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.853055 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ee1375b2eca6432291e041e9aaf7bf3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:56.853381 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee1375b2eca6432291e041e9aaf7bf3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:56.854307 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a74b1be50de04891b89cb1445a48a40e" } new_table_name: "default.Table_5d25b16481c84800a7dbb53c61132cbb" modify_external_catalogs: true
W20251028 02:59:56.855952 8218 consensus_peers.cc:597] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda -> Peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Couldn't send request to peer fbd2afb4894c4e7a8520cc09f8593aa0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0703e18d7dba4d628e3cc310be55fcf5. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:56.856447 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_31e973dfaa9b49f2bed3e9dfd11f7ae9" } modify_external_catalogs: true
I20251028 02:59:56.856535 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_31e973dfaa9b49f2bed3e9dfd11f7ae9" } modify_external_catalogs: true
I20251028 02:59:56.857312 7965 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 2bd4558fc315491687a12efce7ee8ab2 (table default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9]). Expected version 1 got 0
I20251028 02:59:56.857913 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_424dccd707fc4fa1bd9d01e78cab2f92" } modify_external_catalogs: true
I20251028 02:59:56.857997 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_424dccd707fc4fa1bd9d01e78cab2f92" } modify_external_catalogs: true
I20251028 02:59:56.858054 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 929cf41ac7b0437387fbc488983f5cac with delete_type TABLET_DATA_DELETED (replaced by tablet b5e3c89222754eacb6ba4773396d62a9) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.858301 8550 tablet_replica.cc:333] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.858455 8550 raft_consensus.cc:2243] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.858573 8550 raft_consensus.cc:2272] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.859099 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 06302d2cb5554805b85916741ea10f0c (table default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d]). Expected version 2 got 0
I20251028 02:59:56.859179 8550 ts_tablet_manager.cc:1916] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.860266 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8f6fdbd222f4456596107350e6c3c6e1" } modify_external_catalogs: true
I20251028 02:59:56.860364 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8f6fdbd222f4456596107350e6c3c6e1" } modify_external_catalogs: true
I20251028 02:59:56.861212 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 49b0b3459d434cf2810029ce205e745c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.861573 8550 ts_tablet_manager.cc:1929] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.861649 8550 log.cc:1199] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/929cf41ac7b0437387fbc488983f5cac
I20251028 02:59:56.862730 8550 ts_tablet_manager.cc:1950] T 929cf41ac7b0437387fbc488983f5cac P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.863245 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3f04c867f0ce4ab99f980cca063abc30
I20251028 02:59:56.863543 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 929cf41ac7b0437387fbc488983f5cac (table default.Table_aa25a85ac8d24940a727a586d6c630db [id=10f40fa1652f495e91fcbd5852bc3464]) successfully deleted
I20251028 02:59:56.864423 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f04c867f0ce4ab99f980cca063abc30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.864770 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f04c867f0ce4ab99f980cca063abc30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:56.865164 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 3f04c867f0ce4ab99f980cca063abc30 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f04c867f0ce4ab99f980cca063abc30 already in progress: creating tablet
I20251028 02:59:56.865309 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9621e42e6a324ecda50fe83c0c53119c
I20251028 02:59:56.865561 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 9621e42e6a324ecda50fe83c0c53119c deleted and replaced by tablet bedb4fd3b46f4453b194cfaea0c3c701
I20251028 02:59:56.866223 9401 raft_consensus.cc:493] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.866324 9401 raft_consensus.cc:515] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.866690 9401 leader_election.cc:290] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.866245 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 9621e42e6a324ecda50fe83c0c53119c with delete_type TABLET_DATA_DELETED (replaced by tablet bedb4fd3b46f4453b194cfaea0c3c701) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.867018 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 9621e42e6a324ecda50fe83c0c53119c with delete_type TABLET_DATA_DELETED (replaced by tablet bedb4fd3b46f4453b194cfaea0c3c701) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.867170 8281 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fcab55fd49da41f7a95d46d76a8dac8b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.867504 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a9653fc3236f4e0b812de5a859932587
W20251028 02:59:56.867631 8083 leader_election.cc:343] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.867738 7963 catalog_manager.cc:6719] ReplaceTablet: tablet a9653fc3236f4e0b812de5a859932587 deleted and replaced by tablet 58268ac590f3437a942dcafd927a5496
W20251028 02:59:56.868002 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9621e42e6a324ecda50fe83c0c53119c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9621e42e6a324ecda50fe83c0c53119c already in progress: creating tablet
I20251028 02:59:56.868557 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet a9653fc3236f4e0b812de5a859932587 with delete_type TABLET_DATA_DELETED (replaced by tablet 58268ac590f3437a942dcafd927a5496) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.869146 7963 catalog_manager.cc:5649] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:56.869408 7963 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]). Expected version 1 got 0
W20251028 02:59:56.869462 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a9653fc3236f4e0b812de5a859932587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9653fc3236f4e0b812de5a859932587 already in progress: creating tablet
I20251028 02:59:56.870067 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet a9653fc3236f4e0b812de5a859932587 with delete_type TABLET_DATA_DELETED (replaced by tablet 58268ac590f3437a942dcafd927a5496) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.874881 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dab70ca2d9f4b5fba338cb337b0b396. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.875000 8253 tablet_service.cc:1505] Processing CreateTablet for tablet f39ed19223fa4268a01aaf4c7762d86e (DEFAULT_TABLE table=default.Table_4debdeef26b14d9b93a878210eaba4c3 [id=5a391450a3a04f01bcd5c3c06d36cf66]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.875329 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f39ed19223fa4268a01aaf4c7762d86e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:56.880044 8220 consensus_peers.cc:597] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 506396e55fc242ada23d66088984f0ae. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:56.880705 9401 raft_consensus.cc:493] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.880807 9401 raft_consensus.cc:515] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.881096 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 874b6660d86b48ebad03ecc54588da28 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:56.881235 9401 leader_election.cc:290] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.881830 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a344e2e03e4eeaa688275e9ecbf63f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.881978 8284 raft_consensus.cc:2393] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 02:59:56.886646 8552 ts_tablet_manager.cc:1929] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.886742 8552 log.cc:1199] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/73c20437695a4043a86faa9c98091e5e
W20251028 02:59:56.886781 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5428db004de74f7492c9b8999e53020e consensus state has no leader
I20251028 02:59:56.886999 8552 ts_tablet_manager.cc:1950] T 73c20437695a4043a86faa9c98091e5e P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 02:59:56.887120 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5c1994484abf4b6fbbc3f47527558cdb consensus state has no leader
I20251028 02:59:56.887759 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 73c20437695a4043a86faa9c98091e5e (table default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
I20251028 02:59:56.887982 8552 tablet_replica.cc:333] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:56.888126 8552 raft_consensus.cc:2243] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.888221 8552 raft_consensus.cc:2272] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.889540 8552 ts_tablet_manager.cc:1916] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.891588 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet bd50e7d4e28541de9799833c53fe7c02
I20251028 02:59:56.892153 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet bd50e7d4e28541de9799833c53fe7c02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.892647 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9e908d5921d34c9396c1ccdc857c66b7
W20251028 02:59:56.893515 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet bd50e7d4e28541de9799833c53fe7c02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd50e7d4e28541de9799833c53fe7c02 already in progress: creating tablet
I20251028 02:59:56.894762 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 0fcabc4a2fd44043be5451046ff5bd97
I20251028 02:59:56.895280 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e908d5921d34c9396c1ccdc857c66b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.895469 8145 raft_consensus.cc:3060] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.897584 8550 tablet_replica.cc:333] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.897802 8550 raft_consensus.cc:2243] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.897958 8550 raft_consensus.cc:2272] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.898509 8550 ts_tablet_manager.cc:1916] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.901152 9401 raft_consensus.cc:493] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.901289 8550 ts_tablet_manager.cc:1929] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.6
I20251028 02:59:56.901257 9401 raft_consensus.cc:515] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.901345 8550 log.cc:1199] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9e908d5921d34c9396c1ccdc857c66b7
I20251028 02:59:56.901605 8550 ts_tablet_manager.cc:1950] T 9e908d5921d34c9396c1ccdc857c66b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.901715 9401 leader_election.cc:290] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.902215 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 0fcabc4a2fd44043be5451046ff5bd97 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.902660 8254 tablet_service.cc:1552] Processing DeleteTablet for tablet 0fcabc4a2fd44043be5451046ff5bd97 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.902824 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e908d5921d34c9396c1ccdc857c66b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.903450 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9e908d5921d34c9396c1ccdc857c66b7 (table default.Table_8f6fdbd222f4456596107350e6c3c6e1 [id=4eb2743632814baab2e39bbdce700d6b]) successfully deleted
W20251028 02:59:56.903857 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 0fcabc4a2fd44043be5451046ff5bd97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fcabc4a2fd44043be5451046ff5bd97 already in progress: creating tablet
I20251028 02:59:56.904357 8284 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a2ebe28dbd2439e963319ec6e515635" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:56.904507 8284 raft_consensus.cc:2393] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 02:59:56.907584 8546 tablet_bootstrap.cc:492] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.907678 8546 ts_tablet_manager.cc:1403] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.239s user 0.002s sys 0.001s
I20251028 02:59:56.908118 8546 raft_consensus.cc:359] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.908216 8546 raft_consensus.cc:385] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.908245 8546 raft_consensus.cc:740] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.908362 8546 consensus_queue.cc:260] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.908701 8546 ts_tablet_manager.cc:1434] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.908867 8546 tablet_bootstrap.cc:492] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:56.909868 8546 tablet_bootstrap.cc:654] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.912809 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet bd50e7d4e28541de9799833c53fe7c02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:56.922617 7960 catalog_manager.cc:5649] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:56.922870 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 06302d2cb5554805b85916741ea10f0c (table default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d]). Expected version 2 got 0
I20251028 02:59:56.923053 7960 catalog_manager.cc:5649] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:56.793463 8399 tablet_service.cc:1552] Processing DeleteTablet for tablet 26a344e2e03e4eeaa688275e9ecbf63f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.794441 8394 tablet_service.cc:1505] Processing CreateTablet for tablet b7d3062426434c339be4a745decae729 (DEFAULT_TABLE table=default.Table_4045ef6d27fd4f2fbfb8975411994305 [id=1dd28e8d2834401589250978261308c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.923477 8546 tablet_bootstrap.cc:492] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:56.923470 8551 log.cc:1199] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9bf28d9bd3bd462f864fc4f20468e605
I20251028 02:59:56.923573 8546 ts_tablet_manager.cc:1403] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 02:59:56.923738 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7d3062426434c339be4a745decae729. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.923915 8551 ts_tablet_manager.cc:1950] T 9bf28d9bd3bd462f864fc4f20468e605 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:56.924204 8546 raft_consensus.cc:359] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.924314 8546 raft_consensus.cc:385] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.924345 8546 raft_consensus.cc:740] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.925043 8125 tablet_service.cc:1505] Processing CreateTablet for tablet d0dad91f926648c781d1582733d6ae95 (DEFAULT_TABLE table=default.Table_0ae27331d550469cb43063ea5db69a57 [id=d334449caa98411db42e31c49ebbd336]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.925360 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0dad91f926648c781d1582733d6ae95. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.795018 8395 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c41d49f9eba40a48bd6900671297db9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.796594 8545 tablet_bootstrap.cc:492] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.926236 8545 ts_tablet_manager.cc:1403] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.340s user 0.002s sys 0.000s
I20251028 02:59:56.926496 8546 consensus_queue.cc:260] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.926786 8545 raft_consensus.cc:359] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.926862 8551 tablet_replica.cc:333] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:56.926890 8545 raft_consensus.cc:385] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.926927 8545 raft_consensus.cc:740] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.926999 8551 raft_consensus.cc:2243] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.927048 8545 consensus_queue.cc:260] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.927093 8551 raft_consensus.cc:2272] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.927374 8545 ts_tablet_manager.cc:1434] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.927482 8551 ts_tablet_manager.cc:1916] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.927573 8545 tablet_bootstrap.cc:492] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.927906 8552 ts_tablet_manager.cc:1929] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.927973 8552 log.cc:1199] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/bb0e20b3e05c484394e11bbdea6a44ea
I20251028 02:59:56.928184 8552 ts_tablet_manager.cc:1950] T bb0e20b3e05c484394e11bbdea6a44ea P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:56.928610 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9bf28d9bd3bd462f864fc4f20468e605 (table default.Table_ce36e2a0a4374085b1e87288b3288ab9 [id=e61fa366db844911bb80af844de41fa6]) successfully deleted
I20251028 02:59:56.928881 8545 tablet_bootstrap.cc:654] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.797659 9586 consensus_queue.cc:1048] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:56.798317 8400 tablet_service.cc:1505] Processing CreateTablet for tablet ad926e9ee0044c62a7abbfca959a4f56 (DEFAULT_TABLE table=default.Table_6ad31a79d9f043628bdac1dd2d4c263c [id=4570d7ddf33c447fb40d6411ea47e52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.929831 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad926e9ee0044c62a7abbfca959a4f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.929934 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]). Expected version 1 got 0
I20251028 02:59:56.930169 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 06302d2cb5554805b85916741ea10f0c (table default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d]). Expected version 2 got 0
I20251028 02:59:56.931109 8546 ts_tablet_manager.cc:1434] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20251028 02:59:56.798764 8393 tablet_service.cc:1505] Processing CreateTablet for tablet 1c41d49f9eba40a48bd6900671297db9 (DEFAULT_TABLE table=default.Table_46656faddd3141adbdd28b70154be5f4 [id=ab17c384ca9242f094e9373db2ef9c7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.931495 8393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c41d49f9eba40a48bd6900671297db9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.931584 7963 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_f99c9b345302415b87a08fe5c4777310 [id=da79674aa8cd468e84114457ee46766e] alter complete (version 1)
I20251028 02:59:56.931595 8399 tablet_service.cc:1505] Processing CreateTablet for tablet d0dad91f926648c781d1582733d6ae95 (DEFAULT_TABLE table=default.Table_0ae27331d550469cb43063ea5db69a57 [id=d334449caa98411db42e31c49ebbd336]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.931890 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0dad91f926648c781d1582733d6ae95. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.803198 8396 tablet_service.cc:1505] Processing CreateTablet for tablet e3b33c4d189247a89878b6a8e1fcbac0 (DEFAULT_TABLE table=default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.933454 8545 tablet_bootstrap.cc:492] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:56.933830 8545 ts_tablet_manager.cc:1403] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:56.934506 9401 raft_consensus.cc:493] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.934590 9401 raft_consensus.cc:515] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.934697 8545 raft_consensus.cc:359] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.934782 8545 raft_consensus.cc:385] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.934811 8545 raft_consensus.cc:740] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.934990 9401 leader_election.cc:290] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.934959 8545 consensus_queue.cc:260] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.804565 8398 tablet_service.cc:1505] Processing CreateTablet for tablet ba64524142934af08658e434e7fb08ed (DEFAULT_TABLE table=default.Table_8b7624342986474fbd0e31c86947f781 [id=530470c553154d76bf0df08ddfbf2fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.804986 8401 tablet_service.cc:1505] Processing CreateTablet for tablet f39ed19223fa4268a01aaf4c7762d86e (DEFAULT_TABLE table=default.Table_4debdeef26b14d9b93a878210eaba4c3 [id=5a391450a3a04f01bcd5c3c06d36cf66]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.935285 8545 ts_tablet_manager.cc:1434] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:56.829353 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet f6ac7b5fff0848cebf710ba3ed50def4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f90e3c3524e4498aa6c4bfc5304d44b) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.849915 8391 tablet_service.cc:1552] Processing DeleteTablet for tablet 45572222af854b34a5f26a00f8ad9615 with delete_type TABLET_DATA_DELETED (replaced by tablet d60fc654f6af44b290d46235082a4113) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.856670 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet 551aea93ba064bc89ae00b002eb7ccfc with delete_type TABLET_DATA_DELETED (replaced by tablet a4bcdc8a0c554b7cae81d7993e4f2861) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.864382 8389 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f04c867f0ce4ab99f980cca063abc30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.935508 8545 tablet_bootstrap.cc:492] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:56.866106 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet 9621e42e6a324ecda50fe83c0c53119c with delete_type TABLET_DATA_DELETED (replaced by tablet bedb4fd3b46f4453b194cfaea0c3c701) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.867046 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fcab55fd49da41f7a95d46d76a8dac8b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.869254 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet a9653fc3236f4e0b812de5a859932587 with delete_type TABLET_DATA_DELETED (replaced by tablet 58268ac590f3437a942dcafd927a5496) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.884536 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a344e2e03e4eeaa688275e9ecbf63f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.892339 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet bd50e7d4e28541de9799833c53fe7c02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.893318 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e908d5921d34c9396c1ccdc857c66b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.895437 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 0fcabc4a2fd44043be5451046ff5bd97 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.902444 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2a2ebe28dbd2439e963319ec6e515635" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.911435 9475 consensus_queue.cc:1048] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:56.937436 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 04d271c95c2847ca8b1fd2d55741117c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:56.914863 8354 consensus_peers.cc:597] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 -> Peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Couldn't send request to peer fbd2afb4894c4e7a8520cc09f8593aa0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9e908d5921d34c9396c1ccdc857c66b7. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:56.937614 8550 tablet_replica.cc:333] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.937779 8550 raft_consensus.cc:2243] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.937893 8550 raft_consensus.cc:2272] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.938133 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e211a1a8c547baaee1199f7103bfad" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.938457 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]). Expected version 1 got 0
I20251028 02:59:56.815711 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 8dab70ca2d9f4b5fba338cb337b0b396 (DEFAULT_TABLE table=default.Table_f4e5df9294de4390bfbaaf9e86ade7c7 [id=eab0e9b296d04340b7a2fd250f68d9f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.933923 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3b33c4d189247a89878b6a8e1fcbac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.939047 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dab70ca2d9f4b5fba338cb337b0b396. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.939591 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 49b0b3459d434cf2810029ce205e745c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:56.941505 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f39ed19223fa4268a01aaf4c7762d86e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.941825 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba64524142934af08658e434e7fb08ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.941921 8384 tablet_service.cc:1505] Processing CreateTablet for tablet f6818c3f2dfa48448bb2d9b38ed0dc19 (DEFAULT_TABLE table=default.Table_a35599c912a74f959526ab6945536e2d [id=24424b561e55419cbc0d32089a83e21a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.942545 8385 tablet_service.cc:1505] Processing CreateTablet for tablet ee2fd98744ff44028f36bb55e6eba51e (DEFAULT_TABLE table=default.Table_c90ec4690cc24b63ac21b51b090816e5 [id=e77c51294af5486f9c30cd2f2c2a4eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.942579 8254 tablet_service.cc:1505] Processing CreateTablet for tablet ee2fd98744ff44028f36bb55e6eba51e (DEFAULT_TABLE table=default.Table_c90ec4690cc24b63ac21b51b090816e5 [id=e77c51294af5486f9c30cd2f2c2a4eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.942806 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee2fd98744ff44028f36bb55e6eba51e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.945060 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 597d792ed13941a2a895b0dc305614f1 (DEFAULT_TABLE table=default.Table_2adb2ccc4e3e4c1396032eae34eeb18c [id=bf54b516275b46baa498e3b6a5150c97]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.945358 8261 tablet_service.cc:1505] Processing CreateTablet for tablet a38ef10c145a4d00a26265a9dee5c744 (DEFAULT_TABLE table=default.Table_892e6fce625c4e529ee8b9d81330b0d3 [id=ba8b6a629ebc489abe7043890455765a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.945813 8256 tablet_service.cc:1505] Processing CreateTablet for tablet f62b981b9b1a44bb96bbc90f924c3b64 (DEFAULT_TABLE table=default.Table_758c1adf3eb6460d9fa4e675a42507e3 [id=d4eaebd005bc42dd86d7125ef555546f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.946265 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 8483cbc3099644d2b6f7ff52869e21ea (DEFAULT_TABLE table=default.Table_8d295e2ce9374929ab925a8d8329920d [id=de05ccfb39064421868853d6d9866bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.947038 8553 raft_consensus.cc:493] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.947145 8553 raft_consensus.cc:515] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.947485 8397 tablet_service.cc:1505] Processing CreateTablet for tablet f62b981b9b1a44bb96bbc90f924c3b64 (DEFAULT_TABLE table=default.Table_758c1adf3eb6460d9fa4e675a42507e3 [id=d4eaebd005bc42dd86d7125ef555546f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.947585 8553 leader_election.cc:290] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.948083 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49b46c13cb4e46388ed0a69d2ced9b42" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.946283 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6818c3f2dfa48448bb2d9b38ed0dc19. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.948320 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 49b0b3459d434cf2810029ce205e745c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.948482 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 4e44345b09584bbca19b19e57ce17c77 with delete_type TABLET_DATA_DELETED (replaced by tablet c8b4b0aa30194ba495adca9138e2454e) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.948637 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49b46c13cb4e46388ed0a69d2ced9b42" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.948756 8144 raft_consensus.cc:2468] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:56.948923 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 597d792ed13941a2a895b0dc305614f1 (DEFAULT_TABLE table=default.Table_2adb2ccc4e3e4c1396032eae34eeb18c [id=bf54b516275b46baa498e3b6a5150c97]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.948320 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 35e211a1a8c547baaee1199f7103bfad with delete_type TABLET_DATA_DELETED (replaced by tablet 51198eb794764f7fbfda806fc256bcca) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:56.949137 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 597d792ed13941a2a895b0dc305614f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.949221 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 26a344e2e03e4eeaa688275e9ecbf63f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:56.949358 8083 leader_election.cc:343] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.949422 8083 leader_election.cc:304] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:56.949630 8083 leader_election.cc:304] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:56.949782 8562 raft_consensus.cc:2749] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.949884 8562 raft_consensus.cc:3060] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.950947 8562 raft_consensus.cc:2749] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.951161 8218 leader_election.cc:304] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:56.951208 8083 leader_election.cc:304] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:56.951416 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8483cbc3099644d2b6f7ff52869e21ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.951833 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f62b981b9b1a44bb96bbc90f924c3b64. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:56.953516 8220 leader_election.cc:343] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.954272 8553 raft_consensus.cc:2804] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.954329 8553 raft_consensus.cc:493] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.954366 8553 raft_consensus.cc:3060] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.955454 8553 raft_consensus.cc:515] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.955840 8553 leader_election.cc:290] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.956480 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49b46c13cb4e46388ed0a69d2ced9b42" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:56.956460 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49b46c13cb4e46388ed0a69d2ced9b42" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.956583 8144 raft_consensus.cc:3060] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:56.957430 8220 leader_election.cc:343] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:56.957446 8251 tablet_service.cc:1505] Processing CreateTablet for tablet a4bcdc8a0c554b7cae81d7993e4f2861 (DEFAULT_TABLE table=default.Table_806d85836a4c44a986095f5108a93d7f [id=c098c2727dd147e69471353bd9a741fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.957633 8144 raft_consensus.cc:2468] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:56.957733 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 3435803c6167447a8b3afba6c0f3511b (DEFAULT_TABLE table=default.Table_e40f83dab24e4e83b634a88cfe208817 [id=b987718414e246efb3353a3c8bd35f2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.957952 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 8483cbc3099644d2b6f7ff52869e21ea (DEFAULT_TABLE table=default.Table_8d295e2ce9374929ab925a8d8329920d [id=de05ccfb39064421868853d6d9866bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.958011 8218 leader_election.cc:304] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:56.958216 8259 tablet_service.cc:1505] Processing CreateTablet for tablet e9ad4ae17665483584ce2d638d1639fb (DEFAULT_TABLE table=default.Table_929b382a93794515a84b4f6356fb86b2 [id=5f35ae4ce3914890a3ce64198fbdb39e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.958479 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 28b81f0000c4407096feff773c29579d (DEFAULT_TABLE table=default.Table_fa28ec70564a473fb82a04887b59f553 [id=682408eb6dfa4bc28bd0c7b9175a4c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.958729 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 3b0c593a3b96434db9ac42d260fc6997 (DEFAULT_TABLE table=default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.959004 8264 tablet_service.cc:1505] Processing CreateTablet for tablet e369dbabc2c04ad5ab17bbd4225127c1 (DEFAULT_TABLE table=default.Table_621dda8a15654228acd735a8dca754ef [id=36f3b29b4d154e6caf4c3fb52bc82466]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.959110 8385 tablet_service.cc:1505] Processing CreateTablet for tablet e369dbabc2c04ad5ab17bbd4225127c1 (DEFAULT_TABLE table=default.Table_621dda8a15654228acd735a8dca754ef [id=36f3b29b4d154e6caf4c3fb52bc82466]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.959394 8387 tablet_service.cc:1505] Processing CreateTablet for tablet a578008e735a4cc1a2f50dba09c16117 (DEFAULT_TABLE table=default.Table_c4cf0813efcd412a97a6c42da08e8ea9 [id=888eac279afa4ab2939d62fc95bcf66a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.959455 8262 tablet_service.cc:1505] Processing CreateTablet for tablet a578008e735a4cc1a2f50dba09c16117 (DEFAULT_TABLE table=default.Table_c4cf0813efcd412a97a6c42da08e8ea9 [id=888eac279afa4ab2939d62fc95bcf66a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.959247 8265 tablet_service.cc:1505] Processing CreateTablet for tablet 26a60624333d43f1b701036b7e9dfdb0 (DEFAULT_TABLE table=default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.959574 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 26a60624333d43f1b701036b7e9dfdb0 (DEFAULT_TABLE table=default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.959411 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e369dbabc2c04ad5ab17bbd4225127c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.959805 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a60624333d43f1b701036b7e9dfdb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.959839 9401 raft_consensus.cc:3060] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.959954 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a578008e735a4cc1a2f50dba09c16117. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.960142 8401 tablet_service.cc:1505] Processing CreateTablet for tablet a4bcdc8a0c554b7cae81d7993e4f2861 (DEFAULT_TABLE table=default.Table_806d85836a4c44a986095f5108a93d7f [id=c098c2727dd147e69471353bd9a741fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.960350 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4bcdc8a0c554b7cae81d7993e4f2861. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.960911 9401 raft_consensus.cc:2749] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:56.963814 8397 tablet_service.cc:1505] Processing CreateTablet for tablet e9ad4ae17665483584ce2d638d1639fb (DEFAULT_TABLE table=default.Table_929b382a93794515a84b4f6356fb86b2 [id=5f35ae4ce3914890a3ce64198fbdb39e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.964105 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9ad4ae17665483584ce2d638d1639fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.965420 8553 raft_consensus.cc:2804] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.965533 8553 raft_consensus.cc:697] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:56.965680 8553 consensus_queue.cc:237] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.966809 8396 tablet_service.cc:1505] Processing CreateTablet for tablet a38ef10c145a4d00a26265a9dee5c744 (DEFAULT_TABLE table=default.Table_892e6fce625c4e529ee8b9d81330b0d3 [id=ba8b6a629ebc489abe7043890455765a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.967041 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38ef10c145a4d00a26265a9dee5c744. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.967092 8550 ts_tablet_manager.cc:1916] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:56.969656 8250 tablet_service.cc:1505] Processing CreateTablet for tablet 4b787f8f024745b297b2ebaa456cbcbf (DEFAULT_TABLE table=default.Table_35aa9cb3afdc4a149341fa51b840258f [id=a05d3a9fbe8c4278bcab247dcd663653]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.970504 8550 ts_tablet_manager.cc:1929] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.970619 8550 log.cc:1199] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/04d271c95c2847ca8b1fd2d55741117c
I20251028 02:59:56.970892 8550 ts_tablet_manager.cc:1950] T 04d271c95c2847ca8b1fd2d55741117c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:56.971460 8249 tablet_service.cc:1505] Processing CreateTablet for tablet d5c3b02db24d4dc48260b42841844938 (DEFAULT_TABLE table=default.Table_f170e997649f4ff3a51b21a5ee733768 [id=3a8e945f16744123a8583ec0db730658]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.971592 8247 tablet_service.cc:1505] Processing CreateTablet for tablet 58268ac590f3437a942dcafd927a5496 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.971736 8248 tablet_service.cc:1505] Processing CreateTablet for tablet bedb4fd3b46f4453b194cfaea0c3c701 (DEFAULT_TABLE table=default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.971824 8550 tablet_replica.cc:333] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:56.971975 8550 raft_consensus.cc:2243] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:56.972101 8550 raft_consensus.cc:2272] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:56.972190 8246 tablet_service.cc:1505] Processing CreateTablet for tablet 46ae536d269d4cbdbcee44a26d25bd67 (DEFAULT_TABLE table=default.Table_0325897e2eb5414685dd8be9e6d2fd2e [id=b9c19c3aa17e4059b3f58533143edff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.972415 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 28b81f0000c4407096feff773c29579d (DEFAULT_TABLE table=default.Table_fa28ec70564a473fb82a04887b59f553 [id=682408eb6dfa4bc28bd0c7b9175a4c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.972683 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28b81f0000c4407096feff773c29579d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.973915 8551 ts_tablet_manager.cc:1929] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:56.973996 8551 log.cc:1199] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/73c20437695a4043a86faa9c98091e5e
I20251028 02:59:56.974221 8551 ts_tablet_manager.cc:1950] T 73c20437695a4043a86faa9c98091e5e P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:56.975033 8551 tablet_replica.cc:333] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:56.975284 8397 tablet_service.cc:1505] Processing CreateTablet for tablet bedb4fd3b46f4453b194cfaea0c3c701 (DEFAULT_TABLE table=default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.975445 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 46ae536d269d4cbdbcee44a26d25bd67 (DEFAULT_TABLE table=default.Table_0325897e2eb5414685dd8be9e6d2fd2e [id=b9c19c3aa17e4059b3f58533143edff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.975565 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 58268ac590f3437a942dcafd927a5496 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.975565 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bedb4fd3b46f4453b194cfaea0c3c701. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.975816 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 583d6226a5f44d47a5ea4788c2b0d5ce (DEFAULT_TABLE table=default.Table_5a485f2c39204b9b8ed300bb7e924bed [id=f5dcccbdb3e44a00a0169cff89823b0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.975945 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58268ac590f3437a942dcafd927a5496. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.976117 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 583d6226a5f44d47a5ea4788c2b0d5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.976311 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46ae536d269d4cbdbcee44a26d25bd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.976411 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 73c20437695a4043a86faa9c98091e5e (table default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
W20251028 02:59:56.976541 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f2d0dffdf5cc414ab870e0dde75916d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2d0dffdf5cc414ab870e0dde75916d6 already in progress: creating tablet
I20251028 02:59:56.977381 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 3435803c6167447a8b3afba6c0f3511b (DEFAULT_TABLE table=default.Table_e40f83dab24e4e83b634a88cfe208817 [id=b987718414e246efb3353a3c8bd35f2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.977591 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3435803c6167447a8b3afba6c0f3511b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.980923 8544 tablet_bootstrap.cc:492] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.980950 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 04d271c95c2847ca8b1fd2d55741117c (table default.Table_ad3466e031ae40059b8e3ec01c7ed5a9 [id=f00f5e8a1fef48e480906cbf79121e33]) successfully deleted
I20251028 02:59:56.981060 8544 ts_tablet_manager.cc:1403] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.135s user 0.000s sys 0.002s
I20251028 02:59:56.981604 8544 raft_consensus.cc:359] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.981738 8544 raft_consensus.cc:385] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.981781 8544 raft_consensus.cc:740] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.981921 8544 consensus_queue.cc:260] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.982275 8544 ts_tablet_manager.cc:1434] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251028 02:59:56.982318 8398 tablet_service.cc:1505] Processing CreateTablet for tablet ee1d686a0e4c47f6b214224047a55747 (DEFAULT_TABLE table=default.Table_6ebfbcbecf7c4975ac638a06859f32d2 [id=27a9fc6733374573b6370d76769a7653]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.982306 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 8720ea8c68904d57a3fc5c0e94d338ca (DEFAULT_TABLE table=default.Table_8c6b8c4cb0e84fb88487023453581cec [id=375c6744f94644199d8d6f08f21ea021]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.982491 8544 tablet_bootstrap.cc:492] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.982618 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8720ea8c68904d57a3fc5c0e94d338ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.982748 8253 tablet_service.cc:1505] Processing CreateTablet for tablet f6818c3f2dfa48448bb2d9b38ed0dc19 (DEFAULT_TABLE table=default.Table_a35599c912a74f959526ab6945536e2d [id=24424b561e55419cbc0d32089a83e21a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.982856 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 8e202e53053248a7a9cee6a217999489 (DEFAULT_TABLE table=default.Table_87426eb930ad44e5902eb3aee3da0d71 [id=32dae7dc982547979850fa8ad8a80de9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.983088 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e202e53053248a7a9cee6a217999489. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.983152 8553 raft_consensus.cc:493] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.983238 8553 raft_consensus.cc:515] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.984078 8553 leader_election.cc:290] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.983090 8384 tablet_service.cc:1505] Processing CreateTablet for tablet 3b0c593a3b96434db9ac42d260fc6997 (DEFAULT_TABLE table=default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.984562 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b0c593a3b96434db9ac42d260fc6997. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.985015 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 4f90e3c3524e4498aa6c4bfc5304d44b (DEFAULT_TABLE table=default.Table_bd14742bd9f943cc93df997dfecd2719 [id=6f3e689a22ae4160ae7537d955c06a90]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.985128 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "74120decd53d4de19c5f6fedf70ecaae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:56.985250 8420 raft_consensus.cc:2468] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:56.985265 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f90e3c3524e4498aa6c4bfc5304d44b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.985453 8389 tablet_service.cc:1505] Processing CreateTablet for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b (DEFAULT_TABLE table=default.Table_cc716ae9a4cd4b939b2e461342f03331 [id=7375e54ec4fc4747bf8e8db863ac69ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.985555 8220 leader_election.cc:304] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:56.985688 8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.985729 8553 raft_consensus.cc:2804] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:56.985770 8553 raft_consensus.cc:493] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:56.985808 8553 raft_consensus.cc:3060] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.986029 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "74120decd53d4de19c5f6fedf70ecaae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:56.986148 8144 raft_consensus.cc:2468] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:56.986824 8544 tablet_bootstrap.cc:654] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.986804 8553 raft_consensus.cc:515] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.987473 8553 leader_election.cc:290] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:56.987814 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 4e4d980617994db4a188a98bb31eff92 (DEFAULT_TABLE table=default.Table_f657c3fd0e15427e97d689cdc7266b04 [id=41ce30081ce9431a9ab98f11fe994fad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.988018 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "74120decd53d4de19c5f6fedf70ecaae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:56.988121 8144 raft_consensus.cc:3060] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.988220 8544 tablet_bootstrap.cc:492] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:56.988298 8544 ts_tablet_manager.cc:1403] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:56.988749 8544 raft_consensus.cc:359] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.988842 8544 raft_consensus.cc:385] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:56.988874 8544 raft_consensus.cc:740] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:56.989001 8544 consensus_queue.cc:260] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.989127 8144 raft_consensus.cc:2468] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:56.989293 8544 ts_tablet_manager.cc:1434] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:56.989429 8218 leader_election.cc:304] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:56.989595 9190 raft_consensus.cc:493] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.989593 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "74120decd53d4de19c5f6fedf70ecaae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:56.989704 9190 raft_consensus.cc:515] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.989774 8420 raft_consensus.cc:3060] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:56.990051 9190 leader_election.cc:290] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.990228 8553 raft_consensus.cc:2804] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:56.990411 8553 raft_consensus.cc:697] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:56.989485 8544 tablet_bootstrap.cc:492] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:56.990594 8553 consensus_queue.cc:237] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:56.990715 8420 raft_consensus.cc:2468] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:56.991932 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 597d792ed13941a2a895b0dc305614f1 (DEFAULT_TABLE table=default.Table_2adb2ccc4e3e4c1396032eae34eeb18c [id=bf54b516275b46baa498e3b6a5150c97]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.992106 8544 tablet_bootstrap.cc:654] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:56.992194 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 597d792ed13941a2a895b0dc305614f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.992409 8116 tablet_service.cc:1505] Processing CreateTablet for tablet f6818c3f2dfa48448bb2d9b38ed0dc19 (DEFAULT_TABLE table=default.Table_a35599c912a74f959526ab6945536e2d [id=24424b561e55419cbc0d32089a83e21a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.992614 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6818c3f2dfa48448bb2d9b38ed0dc19. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.994524 8126 tablet_service.cc:1505] Processing CreateTablet for tablet ee2fd98744ff44028f36bb55e6eba51e (DEFAULT_TABLE table=default.Table_c90ec4690cc24b63ac21b51b090816e5 [id=e77c51294af5486f9c30cd2f2c2a4eec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.994648 8117 tablet_service.cc:1505] Processing CreateTablet for tablet f62b981b9b1a44bb96bbc90f924c3b64 (DEFAULT_TABLE table=default.Table_758c1adf3eb6460d9fa4e675a42507e3 [id=d4eaebd005bc42dd86d7125ef555546f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.994735 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee2fd98744ff44028f36bb55e6eba51e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.994663 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 8483cbc3099644d2b6f7ff52869e21ea (DEFAULT_TABLE table=default.Table_8d295e2ce9374929ab925a8d8329920d [id=de05ccfb39064421868853d6d9866bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.994892 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f62b981b9b1a44bb96bbc90f924c3b64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.994989 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8483cbc3099644d2b6f7ff52869e21ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.995081 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 3435803c6167447a8b3afba6c0f3511b (DEFAULT_TABLE table=default.Table_e40f83dab24e4e83b634a88cfe208817 [id=b987718414e246efb3353a3c8bd35f2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.995327 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3435803c6167447a8b3afba6c0f3511b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.997457 8120 tablet_service.cc:1505] Processing CreateTablet for tablet e369dbabc2c04ad5ab17bbd4225127c1 (DEFAULT_TABLE table=default.Table_621dda8a15654228acd735a8dca754ef [id=36f3b29b4d154e6caf4c3fb52bc82466]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.997519 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 3b0c593a3b96434db9ac42d260fc6997 (DEFAULT_TABLE table=default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:56.997660 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e369dbabc2c04ad5ab17bbd4225127c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.997817 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b0c593a3b96434db9ac42d260fc6997. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:56.998911 8562 raft_consensus.cc:493] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.998988 8562 raft_consensus.cc:515] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.999373 8562 leader_election.cc:290] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:56.999456 8562 raft_consensus.cc:493] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:56.999511 8562 raft_consensus.cc:515] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:56.999783 8562 leader_election.cc:290] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.000068 8119 tablet_service.cc:1505] Processing CreateTablet for tablet 4b787f8f024745b297b2ebaa456cbcbf (DEFAULT_TABLE table=default.Table_35aa9cb3afdc4a149341fa51b840258f [id=a05d3a9fbe8c4278bcab247dcd663653]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.000111 8117 tablet_service.cc:1505] Processing CreateTablet for tablet a4bcdc8a0c554b7cae81d7993e4f2861 (DEFAULT_TABLE table=default.Table_806d85836a4c44a986095f5108a93d7f [id=c098c2727dd147e69471353bd9a741fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.000159 8123 tablet_service.cc:1505] Processing CreateTablet for tablet bedb4fd3b46f4453b194cfaea0c3c701 (DEFAULT_TABLE table=default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.000325 8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b787f8f024745b297b2ebaa456cbcbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.000447 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bedb4fd3b46f4453b194cfaea0c3c701. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.000591 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4bcdc8a0c554b7cae81d7993e4f2861. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.000626 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee1d686a0e4c47f6b214224047a55747. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.000654 8127 tablet_service.cc:1505] Processing CreateTablet for tablet d5c3b02db24d4dc48260b42841844938 (DEFAULT_TABLE table=default.Table_f170e997649f4ff3a51b21a5ee733768 [id=3a8e945f16744123a8583ec0db730658]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.000878 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5c3b02db24d4dc48260b42841844938. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.002641 8123 tablet_service.cc:1505] Processing CreateTablet for tablet 58268ac590f3437a942dcafd927a5496 (DEFAULT_TABLE table=default.Table_dcbff39c1ba04e408b0a0c2ee1f42547 [id=fdb289a561dd4c6497173ef43e2e9ccd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.002743 9475 consensus_queue.cc:1048] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.002848 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58268ac590f3437a942dcafd927a5496. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.003026 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 26a60624333d43f1b701036b7e9dfdb0 (DEFAULT_TABLE table=default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.003152 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 6dd563e13e624d4bb88950b46d480876 (DEFAULT_TABLE table=default.Table_ee1375b2eca6432291e041e9aaf7bf3a [id=eaebbf79fab746dfaa81407d3509306c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.003250 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a60624333d43f1b701036b7e9dfdb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.003026 8115 tablet_service.cc:1505] Processing CreateTablet for tablet d60fc654f6af44b290d46235082a4113 (DEFAULT_TABLE table=default.Table_dbd811be59344e8587fff44779930d85 [id=94895dadf0e34a19a454e8755c61c785]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.003592 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 583d6226a5f44d47a5ea4788c2b0d5ce (DEFAULT_TABLE table=default.Table_5a485f2c39204b9b8ed300bb7e924bed [id=f5dcccbdb3e44a00a0169cff89823b0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.003683 8121 tablet_service.cc:1505] Processing CreateTablet for tablet ee1d686a0e4c47f6b214224047a55747 (DEFAULT_TABLE table=default.Table_6ebfbcbecf7c4975ac638a06859f32d2 [id=27a9fc6733374573b6370d76769a7653]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.004046 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 8720ea8c68904d57a3fc5c0e94d338ca (DEFAULT_TABLE table=default.Table_8c6b8c4cb0e84fb88487023453581cec [id=375c6744f94644199d8d6f08f21ea021]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.004068 8122 tablet_service.cc:1505] Processing CreateTablet for tablet e9ad4ae17665483584ce2d638d1639fb (DEFAULT_TABLE table=default.Table_929b382a93794515a84b4f6356fb86b2 [id=5f35ae4ce3914890a3ce64198fbdb39e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.004441 8116 tablet_service.cc:1505] Processing CreateTablet for tablet a38ef10c145a4d00a26265a9dee5c744 (DEFAULT_TABLE table=default.Table_892e6fce625c4e529ee8b9d81330b0d3 [id=ba8b6a629ebc489abe7043890455765a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.004602 8114 tablet_service.cc:1505] Processing CreateTablet for tablet 4e4d980617994db4a188a98bb31eff92 (DEFAULT_TABLE table=default.Table_f657c3fd0e15427e97d689cdc7266b04 [id=41ce30081ce9431a9ab98f11fe994fad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.004727 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 8e202e53053248a7a9cee6a217999489 (DEFAULT_TABLE table=default.Table_87426eb930ad44e5902eb3aee3da0d71 [id=32dae7dc982547979850fa8ad8a80de9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.005043 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8720ea8c68904d57a3fc5c0e94d338ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.005170 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 28b81f0000c4407096feff773c29579d (DEFAULT_TABLE table=default.Table_fa28ec70564a473fb82a04887b59f553 [id=682408eb6dfa4bc28bd0c7b9175a4c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.005158 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 583d6226a5f44d47a5ea4788c2b0d5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.005493 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 4f90e3c3524e4498aa6c4bfc5304d44b (DEFAULT_TABLE table=default.Table_bd14742bd9f943cc93df997dfecd2719 [id=6f3e689a22ae4160ae7537d955c06a90]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.005599 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee1d686a0e4c47f6b214224047a55747. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.005718 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e202e53053248a7a9cee6a217999489. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.005924 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f90e3c3524e4498aa6c4bfc5304d44b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.006039 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d60fc654f6af44b290d46235082a4113. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.006147 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b (DEFAULT_TABLE table=default.Table_cc716ae9a4cd4b939b2e461342f03331 [id=7375e54ec4fc4747bf8e8db863ac69ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.006358 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.007357 8114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4d980617994db4a188a98bb31eff92. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.007540 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd563e13e624d4bb88950b46d480876. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.009829 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 46ae536d269d4cbdbcee44a26d25bd67 (DEFAULT_TABLE table=default.Table_0325897e2eb5414685dd8be9e6d2fd2e [id=b9c19c3aa17e4059b3f58533143edff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.010064 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46ae536d269d4cbdbcee44a26d25bd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.011557 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38ef10c145a4d00a26265a9dee5c744. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.012892 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9ad4ae17665483584ce2d638d1639fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.013397 9401 raft_consensus.cc:493] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.013474 9401 raft_consensus.cc:515] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.013867 9401 leader_election.cc:290] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.014190 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4d980617994db4a188a98bb31eff92. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.014478 8126 tablet_service.cc:1505] Processing CreateTablet for tablet a578008e735a4cc1a2f50dba09c16117 (DEFAULT_TABLE table=default.Table_c4cf0813efcd412a97a6c42da08e8ea9 [id=888eac279afa4ab2939d62fc95bcf66a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.014712 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a578008e735a4cc1a2f50dba09c16117. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.016469 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 551aea93ba064bc89ae00b002eb7ccfc with delete_type TABLET_DATA_DELETED (replaced by tablet a4bcdc8a0c554b7cae81d7993e4f2861) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.016222 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fa28ec70564a473fb82a04887b59f553" } modify_external_catalogs: true
I20251028 02:59:57.017552 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fa28ec70564a473fb82a04887b59f553" } modify_external_catalogs: true
I20251028 02:59:57.018926 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28b81f0000c4407096feff773c29579d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.019587 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 2bd4558fc315491687a12efce7ee8ab2 (table default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9]). Expected version 1 got 0
I20251028 02:59:57.020390 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 6dd563e13e624d4bb88950b46d480876 (DEFAULT_TABLE table=default.Table_ee1375b2eca6432291e041e9aaf7bf3a [id=eaebbf79fab746dfaa81407d3509306c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.020679 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd563e13e624d4bb88950b46d480876. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.023350 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 4b787f8f024745b297b2ebaa456cbcbf (DEFAULT_TABLE table=default.Table_35aa9cb3afdc4a149341fa51b840258f [id=a05d3a9fbe8c4278bcab247dcd663653]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.023505 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49b0b3459d434cf2810029ce205e745c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:57.023644 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b787f8f024745b297b2ebaa456cbcbf. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:57.023902 8354 leader_election.cc:343] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.024786 9401 raft_consensus.cc:493] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.024888 9401 raft_consensus.cc:515] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.025215 9401 leader_election.cc:290] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.026316 8386 tablet_service.cc:1505] Processing CreateTablet for tablet d5c3b02db24d4dc48260b42841844938 (DEFAULT_TABLE table=default.Table_f170e997649f4ff3a51b21a5ee733768 [id=3a8e945f16744123a8583ec0db730658]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.026584 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5c3b02db24d4dc48260b42841844938. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.027961 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bd14742bd9f943cc93df997dfecd2719" } new_table_name: "default.Table_42f89d7fccbf4c738293e9796be235a0" modify_external_catalogs: true
I20251028 02:59:57.029129 8257 tablet_service.cc:1505] Processing CreateTablet for tablet d0dad91f926648c781d1582733d6ae95 (DEFAULT_TABLE table=default.Table_0ae27331d550469cb43063ea5db69a57 [id=d334449caa98411db42e31c49ebbd336]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.029858 8545 tablet_bootstrap.cc:654] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.031347 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 28b81f0000c4407096feff773c29579d
I20251028 02:59:57.031459 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e44345b09584bbca19b19e57ce17c77" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.032210 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cbecb6472dc54989beca91ca7bcb0b57" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.033138 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fef84b69b1b746d4b4feeed96670c67b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.033272 8416 raft_consensus.cc:2393] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
W20251028 02:59:57.033620 8083 leader_election.cc:343] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:57.033773 8083 leader_election.cc:343] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.035012 8545 tablet_bootstrap.cc:492] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:57.035163 8545 ts_tablet_manager.cc:1403] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.100s user 0.000s sys 0.002s
W20251028 02:59:57.035364 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4f90e3c3524e4498aa6c4bfc5304d44b consensus state has no leader
I20251028 02:59:57.035382 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 28b81f0000c4407096feff773c29579d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.036074 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a290b8114844aa89b33c75fffc8e1e7" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.036396 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet cbecb6472dc54989beca91ca7bcb0b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.036545 8545 raft_consensus.cc:359] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.036705 8545 raft_consensus.cc:385] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.036792 8545 raft_consensus.cc:740] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.036967 8545 consensus_queue.cc:260] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.038021 9401 raft_consensus.cc:493] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.038132 9401 raft_consensus.cc:515] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.038509 9401 leader_election.cc:290] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.038734 8550 ts_tablet_manager.cc:1916] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.039685 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a290b8114844aa89b33c75fffc8e1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.041002 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "369397a89baa472285b25acd6c78e7ec" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.041280 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 28b81f0000c4407096feff773c29579d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.041783 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_831a31c4a8fe414e8c9e8462b95a612e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.042160 8550 ts_tablet_manager.cc:1929] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.042227 8550 log.cc:1199] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/49b0b3459d434cf2810029ce205e745c
W20251028 02:59:57.042238 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_831a31c4a8fe414e8c9e8462b95a612e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.042440 8550 ts_tablet_manager.cc:1950] T 49b0b3459d434cf2810029ce205e745c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.043017 8550 tablet_replica.cc:333] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.043160 8550 raft_consensus.cc:2243] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.043890 8544 tablet_bootstrap.cc:492] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.043972 8544 ts_tablet_manager.cc:1403] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.055s user 0.000s sys 0.002s
I20251028 02:59:57.044607 8544 raft_consensus.cc:359] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.044809 8544 raft_consensus.cc:385] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.044903 8544 raft_consensus.cc:740] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.045117 8544 consensus_queue.cc:260] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.045235 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d5216511b6cb4223afa3920eec378c6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.045589 8544 ts_tablet_manager.cc:1434] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20251028 02:59:57.045676 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d5216511b6cb4223afa3920eec378c6b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.046622 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_fcb100729f5b48b990a9c4d2d9b9fcf2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.046979 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fcb100729f5b48b990a9c4d2d9b9fcf2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.047936 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f4e5df9294de4390bfbaaf9e86ade7c7" } new_table_name: "default.Table_cff7897b7440432c96a505a493a0b149" modify_external_catalogs: true
I20251028 02:59:57.048144 8544 tablet_bootstrap.cc:492] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.048919 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_316b63d9aec64a06b194dca118966b47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.049254 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_316b63d9aec64a06b194dca118966b47 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.049561 8544 tablet_bootstrap.cc:654] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.050276 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 2893b3a6e8854187b837a7e6fef58c4e from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.050654 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_dca026923a844b5faf18d96820d52dbd" } new_table_name: "default.Table_f0dafbe2e0a84541a79cf6c887d59619" modify_external_catalogs: true
I20251028 02:59:57.051782 8545 ts_tablet_manager.cc:1434] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20251028 02:59:57.052259 8272 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fef84b69b1b746d4b4feeed96670c67b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.052528 8279 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49b0b3459d434cf2810029ce205e745c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.052670 8279 raft_consensus.cc:2468] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:57.053086 8281 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e211a1a8c547baaee1199f7103bfad" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.053205 8281 raft_consensus.cc:2468] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:57.053341 8275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "cbecb6472dc54989beca91ca7bcb0b57" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.053689 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_62c4ca6858344881a02383c7919d85cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.053839 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e44345b09584bbca19b19e57ce17c77" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.053930 8274 raft_consensus.cc:2468] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:57.054069 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "369397a89baa472285b25acd6c78e7ec" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:57.054095 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_62c4ca6858344881a02383c7919d85cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.054176 8271 raft_consensus.cc:2393] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 02:59:57.052525 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a290b8114844aa89b33c75fffc8e1e7" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.062284 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]). Expected version 2 got 0
I20251028 02:59:57.063616 8083 leader_election.cc:304] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.065629 7964 catalog_manager.cc:5649] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:57.065920 7964 catalog_manager.cc:5649] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:57.066062 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]). Expected version 1 got 0
I20251028 02:59:57.066841 9475 consensus_queue.cc:1048] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.067323 8356 leader_election.cc:304] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:57.067499 9475 raft_consensus.cc:2804] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:57.067553 9475 raft_consensus.cc:493] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:57.067596 9475 raft_consensus.cc:3060] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.068766 9475 raft_consensus.cc:515] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.069159 9475 leader_election.cc:290] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.069216 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 49b0b3459d434cf2810029ce205e745c (table default.Table_75e654d01f4147b3b7d928c99967c5cc [id=f9132fe51a804151b8df8e1e8fc0d220]) successfully deleted
I20251028 02:59:57.070564 9475 consensus_queue.cc:1048] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.070991 9190 consensus_queue.cc:1048] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.071236 9190 consensus_queue.cc:1048] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.071967 9190 consensus_queue.cc:1048] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.073280 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49b0b3459d434cf2810029ce205e745c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:57.073405 8270 raft_consensus.cc:3060] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.074101 9190 consensus_queue.cc:1048] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.074604 8270 raft_consensus.cc:2468] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:57.076925 8356 leader_election.cc:304] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
W20251028 02:59:57.077365 8354 leader_election.cc:343] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 49b0b3459d434cf2810029ce205e745c
I20251028 02:59:57.077852 8083 leader_election.cc:304] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.077908 8562 raft_consensus.cc:3060] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:57.078137 8083 leader_election.cc:343] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.078192 8083 leader_election.cc:304] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.076825 8148 raft_consensus.cc:3060] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.079041 8562 raft_consensus.cc:2749] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.079290 8562 raft_consensus.cc:2804] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251028 02:59:57.079363 8562 raft_consensus.cc:493] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d7e4cf694f71438298dd32a346abcf75)
I20251028 02:59:57.079416 8562 raft_consensus.cc:3060] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Advancing to term 2
I20251028 02:59:57.079771 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]). Expected version 2 got 0
I20251028 02:59:57.080341 8562 raft_consensus.cc:515] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.080577 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bc8cc7f063644f3eaa9fb0abce91091a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.080652 9190 raft_consensus.cc:2804] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:57.080722 8562 leader_election.cc:290] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 2 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.080751 9190 raft_consensus.cc:697] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:57.080808 8562 raft_consensus.cc:2749] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.080904 9190 consensus_queue.cc:237] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:57.081000 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc8cc7f063644f3eaa9fb0abce91091a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.081591 9190 consensus_queue.cc:1048] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.083653 8552 tablet_replica.cc:333] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 02:59:57.083658 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8dab70ca2d9f4b5fba338cb337b0b396 consensus state has no leader
I20251028 02:59:57.083947 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_8249c2c329814e99a3636633a900f8bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.084342 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8249c2c329814e99a3636633a900f8bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.084801 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38ef10c145a4d00a26265a9dee5c744. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.085076 8552 raft_consensus.cc:2243] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.085302 8552 raft_consensus.cc:2272] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.085364 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8d295e2ce9374929ab925a8d8329920d" } modify_external_catalogs: true
I20251028 02:59:57.085460 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8d295e2ce9374929ab925a8d8329920d" } modify_external_catalogs: true
I20251028 02:59:57.085683 8552 ts_tablet_manager.cc:1916] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.086481 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet 6dd563e13e624d4bb88950b46d480876 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.087471 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f62b981b9b1a44bb96bbc90f924c3b64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.087518 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 0e19a2bdc7024b0c9fe84f8983f9d292 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.088819 8552 ts_tablet_manager.cc:1929] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:57.088892 8552 log.cc:1199] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/0703e18d7dba4d628e3cc310be55fcf5
I20251028 02:59:57.088856 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0ce46dd2a59848399a1d4fdeac7deef9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.089164 8552 ts_tablet_manager.cc:1950] T 0703e18d7dba4d628e3cc310be55fcf5 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 02:59:57.089231 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ce46dd2a59848399a1d4fdeac7deef9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.089497 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4bcdc8a0c554b7cae81d7993e4f2861. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.089771 8248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bedb4fd3b46f4453b194cfaea0c3c701. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.089964 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 597d792ed13941a2a895b0dc305614f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.090131 8246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46ae536d269d4cbdbcee44a26d25bd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.090467 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9ad4ae17665483584ce2d638d1639fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.092329 8246 tablet_service.cc:1505] Processing CreateTablet for tablet d60fc654f6af44b290d46235082a4113 (DEFAULT_TABLE table=default.Table_dbd811be59344e8587fff44779930d85 [id=94895dadf0e34a19a454e8755c61c785]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.092577 8246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d60fc654f6af44b290d46235082a4113. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.092597 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 6dd563e13e624d4bb88950b46d480876 (DEFAULT_TABLE table=default.Table_ee1375b2eca6432291e041e9aaf7bf3a [id=eaebbf79fab746dfaa81407d3509306c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.092823 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dd563e13e624d4bb88950b46d480876. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.093796 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28b81f0000c4407096feff773c29579d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.094165 8544 tablet_bootstrap.cc:492] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.094255 8544 ts_tablet_manager.cc:1403] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.046s user 0.001s sys 0.000s
I20251028 02:59:57.094514 8246 tablet_service.cc:1505] Processing CreateTablet for tablet 8720ea8c68904d57a3fc5c0e94d338ca (DEFAULT_TABLE table=default.Table_8c6b8c4cb0e84fb88487023453581cec [id=375c6744f94644199d8d6f08f21ea021]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.094724 8246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8720ea8c68904d57a3fc5c0e94d338ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.094914 8544 raft_consensus.cc:359] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.095037 8544 raft_consensus.cc:385] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.095070 8544 raft_consensus.cc:740] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.095206 8544 consensus_queue.cc:260] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.095595 8250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b787f8f024745b297b2ebaa456cbcbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.095628 8544 ts_tablet_manager.cc:1434] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.095914 8544 tablet_bootstrap.cc:492] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.095979 8263 tablet_service.cc:1505] Processing CreateTablet for tablet ee1d686a0e4c47f6b214224047a55747 (DEFAULT_TABLE table=default.Table_6ebfbcbecf7c4975ac638a06859f32d2 [id=27a9fc6733374573b6370d76769a7653]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.096179 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee1d686a0e4c47f6b214224047a55747. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.097855 8250 tablet_service.cc:1505] Processing CreateTablet for tablet 4e4d980617994db4a188a98bb31eff92 (DEFAULT_TABLE table=default.Table_f657c3fd0e15427e97d689cdc7266b04 [id=41ce30081ce9431a9ab98f11fe994fad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.098068 8250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e4d980617994db4a188a98bb31eff92. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.098235 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 8e202e53053248a7a9cee6a217999489 (DEFAULT_TABLE table=default.Table_87426eb930ad44e5902eb3aee3da0d71 [id=32dae7dc982547979850fa8ad8a80de9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.098459 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e202e53053248a7a9cee6a217999489. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.098521 8544 tablet_bootstrap.cc:654] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.099277 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b0c593a3b96434db9ac42d260fc6997. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.101289 8248 tablet_service.cc:1505] Processing CreateTablet for tablet 4f90e3c3524e4498aa6c4bfc5304d44b (DEFAULT_TABLE table=default.Table_bd14742bd9f943cc93df997dfecd2719 [id=6f3e689a22ae4160ae7537d955c06a90]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.101501 8248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f90e3c3524e4498aa6c4bfc5304d44b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.101467 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b (DEFAULT_TABLE table=default.Table_cc716ae9a4cd4b939b2e461342f03331 [id=7375e54ec4fc4747bf8e8db863ac69ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.101683 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.102164 8250 tablet_service.cc:1552] Processing DeleteTablet for tablet 28b81f0000c4407096feff773c29579d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.102284 8250 tablet_service.cc:1552] Processing DeleteTablet for tablet 4e44345b09584bbca19b19e57ce17c77 with delete_type TABLET_DATA_DELETED (replaced by tablet c8b4b0aa30194ba495adca9138e2454e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.102387 8250 tablet_service.cc:1552] Processing DeleteTablet for tablet 04d271c95c2847ca8b1fd2d55741117c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.103448 8264 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e369dbabc2c04ad5ab17bbd4225127c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.103951 8083 leader_election.cc:304] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
W20251028 02:59:57.104260 8083 leader_election.cc:343] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:57.104408 8083 leader_election.cc:343] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.104463 8083 leader_election.cc:304] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.104647 8083 leader_election.cc:304] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.105438 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e211a1a8c547baaee1199f7103bfad" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:57.105440 8262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a578008e735a4cc1a2f50dba09c16117. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.106148 8546 tablet_bootstrap.cc:492] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.107219 8546 tablet_bootstrap.cc:654] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.108325 8254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee2fd98744ff44028f36bb55e6eba51e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.108466 9646 raft_consensus.cc:2749] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.108583 9646 raft_consensus.cc:2749] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.110666 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3435803c6167447a8b3afba6c0f3511b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.112885 8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58268ac590f3437a942dcafd927a5496. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.114928 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26a60624333d43f1b701036b7e9dfdb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.116932 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8483cbc3099644d2b6f7ff52869e21ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.118248 8544 tablet_bootstrap.cc:492] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.118345 8544 ts_tablet_manager.cc:1403] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20251028 02:59:57.118906 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5c3b02db24d4dc48260b42841844938. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.118854 8544 raft_consensus.cc:359] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.118968 8544 raft_consensus.cc:385] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.118999 8544 raft_consensus.cc:740] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.043265 8550 raft_consensus.cc:2272] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.119153 8562 raft_consensus.cc:2706] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Received pre-election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopped
I20251028 02:59:57.119525 8544 consensus_queue.cc:260] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.119932 8544 ts_tablet_manager.cc:1434] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:57.120182 8544 tablet_bootstrap.cc:492] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.121209 8544 tablet_bootstrap.cc:654] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.122300 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6818c3f2dfa48448bb2d9b38ed0dc19. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.124737 8550 ts_tablet_manager.cc:1916] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.127089 8550 ts_tablet_manager.cc:1929] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.127156 8550 log.cc:1199] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4e44345b09584bbca19b19e57ce17c77
I20251028 02:59:57.127396 8550 ts_tablet_manager.cc:1950] T 4e44345b09584bbca19b19e57ce17c77 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.128094 8550 tablet_replica.cc:333] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.128257 8550 raft_consensus.cc:2243] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.128360 8550 pending_rounds.cc:70] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 02:59:57.128399 8550 pending_rounds.cc:77] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597143682523136 op_type: NO_OP noop_request { }
I20251028 02:59:57.128468 8550 raft_consensus.cc:2889] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:57.128530 8550 raft_consensus.cc:2272] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.129506 8552 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46256 (request call id 573) took 1063 ms. Trace:
I20251028 02:59:57.129594 8552 rpcz_store.cc:276] 1028 02:59:56.066135 (+ 0us) service_pool.cc:168] Inserting onto call queue
1028 02:59:56.072338 (+ 6203us) service_pool.cc:225] Handling call
1028 02:59:56.928806 (+856468us) ts_tablet_manager.cc:1151] Deleting tablet 0703e18d7dba4d628e3cc310be55fcf5
1028 02:59:56.928816 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1028 02:59:57.087075 (+158259us) tablet_metadata.cc:641] Metadata flushed
1028 02:59:57.088773 (+ 1698us) tablet_metadata.cc:641] Metadata flushed
1028 02:59:57.129491 (+ 40718us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":856329}
I20251028 02:59:57.129863 8552 tablet_replica.cc:333] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.129994 8552 raft_consensus.cc:2243] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.130182 8552 pending_rounds.cc:70] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:57.130213 8552 pending_rounds.cc:77] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597143253630976 op_type: NO_OP noop_request { }
I20251028 02:59:57.130280 8552 raft_consensus.cc:2889] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:57.130329 8552 raft_consensus.cc:2272] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.130910 8552 ts_tablet_manager.cc:1916] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:57.131124 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 900973d832c24b62a4554ae9a219cfd8 consensus state has no leader
I20251028 02:59:57.131412 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e0b6845d183a4c93b6f02c99691cac6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.131805 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0b6845d183a4c93b6f02c99691cac6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.133185 8552 ts_tablet_manager.cc:1929] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.133337 8552 log.cc:1199] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/506396e55fc242ada23d66088984f0ae
I20251028 02:59:57.133677 8552 ts_tablet_manager.cc:1950] T 506396e55fc242ada23d66088984f0ae P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.133867 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0dad91f926648c781d1582733d6ae95. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.134294 8552 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46256 (request call id 574) took 1054 ms. Trace:
I20251028 02:59:57.134402 8552 rpcz_store.cc:276] 1028 02:59:56.079823 (+ 0us) service_pool.cc:168] Inserting onto call queue
1028 02:59:56.080304 (+ 481us) service_pool.cc:225] Handling call
1028 02:59:57.129821 (+1049517us) ts_tablet_manager.cc:1151] Deleting tablet 506396e55fc242ada23d66088984f0ae
1028 02:59:57.129828 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1028 02:59:57.132183 (+ 2355us) tablet_metadata.cc:641] Metadata flushed
1028 02:59:57.133131 (+ 948us) tablet_metadata.cc:641] Metadata flushed
1028 02:59:57.134281 (+ 1150us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1049370}
I20251028 02:59:57.134809 8552 tablet_replica.cc:333] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.134999 8552 raft_consensus.cc:2243] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.135334 8552 raft_consensus.cc:2272] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.135816 8552 ts_tablet_manager.cc:1916] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.136006 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 0703e18d7dba4d628e3cc310be55fcf5 (table default.Table_758c1adf3eb6460d9fa4e675a42507e3 [id=d4eaebd005bc42dd86d7125ef555546f]) successfully deleted
I20251028 02:59:57.136183 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 506396e55fc242ada23d66088984f0ae (table default.Table_ba1740929d404081b7451d23bc910475 [id=d41b990640fd4401a2d09310961cbba4]) successfully deleted
I20251028 02:59:57.137794 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_033167c60f644376b85c64d13503579f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.138227 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_033167c60f644376b85c64d13503579f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.139099 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f99c9b345302415b87a08fe5c4777310" } new_table_name: "default.Table_19f5792cad7f4b2fa7f0446f6c3c7bb0" modify_external_catalogs: true
I20251028 02:59:57.139955 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2893b3a6e8854187b837a7e6fef58c4e
I20251028 02:59:57.140205 7965 catalog_manager.cc:6719] ReplaceTablet: tablet 2893b3a6e8854187b837a7e6fef58c4e deleted and replaced by tablet f5e7e0614af04cc2a22130906d062a79
I20251028 02:59:57.140311 8552 ts_tablet_manager.cc:1929] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:57.140395 8552 log.cc:1199] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4ff8c974231f4b8aa18c18a05ad3e738
I20251028 02:59:57.140389 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2fd19ddb9673469a98456a4520898183" } new_table_name: "default.Table_2f27286185dc41e4b5bc773a4ccdd68a" modify_external_catalogs: true
I20251028 02:59:57.140657 8552 ts_tablet_manager.cc:1950] T 4ff8c974231f4b8aa18c18a05ad3e738 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.141180 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 2893b3a6e8854187b837a7e6fef58c4e with delete_type TABLET_DATA_DELETED (replaced by tablet f5e7e0614af04cc2a22130906d062a79) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.141211 8552 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46256 (request call id 575) took 1019 ms. Trace:
I20251028 02:59:57.141250 8552 rpcz_store.cc:276] 1028 02:59:56.121290 (+ 0us) service_pool.cc:168] Inserting onto call queue
1028 02:59:56.121393 (+ 103us) service_pool.cc:225] Handling call
1028 02:59:57.134768 (+1013375us) ts_tablet_manager.cc:1151] Deleting tablet 4ff8c974231f4b8aa18c18a05ad3e738
1028 02:59:57.134786 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
1028 02:59:57.137855 (+ 3069us) tablet_metadata.cc:641] Metadata flushed
1028 02:59:57.140258 (+ 2403us) tablet_metadata.cc:641] Metadata flushed
1028 02:59:57.141203 (+ 945us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1013218}
I20251028 02:59:57.141553 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 2893b3a6e8854187b837a7e6fef58c4e with delete_type TABLET_DATA_DELETED (replaced by tablet f5e7e0614af04cc2a22130906d062a79) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.141598 8552 tablet_replica.cc:333] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.141738 8552 raft_consensus.cc:2243] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.141803 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8483cbc3099644d2b6f7ff52869e21ea
I20251028 02:59:57.141865 8552 raft_consensus.cc:2272] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.142040 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e40f83dab24e4e83b634a88cfe208817" } modify_external_catalogs: true
I20251028 02:59:57.142136 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e40f83dab24e4e83b634a88cfe208817" } modify_external_catalogs: true
I20251028 02:59:57.142160 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_17aa90b035264d2585988c8243238e7a" } modify_external_catalogs: true
I20251028 02:59:57.142243 8552 ts_tablet_manager.cc:1916] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.142254 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_17aa90b035264d2585988c8243238e7a" } modify_external_catalogs: true
I20251028 02:59:57.142536 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 0703e18d7dba4d628e3cc310be55fcf5 with delete_type TABLET_DATA_DELETED (replaced by tablet f62b981b9b1a44bb96bbc90f924c3b64) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.142913 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2adb2ccc4e3e4c1396032eae34eeb18c" } modify_external_catalogs: true
I20251028 02:59:57.130216 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 583d6226a5f44d47a5ea4788c2b0d5ce (DEFAULT_TABLE table=default.Table_5a485f2c39204b9b8ed300bb7e924bed [id=f5dcccbdb3e44a00a0169cff89823b0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.142997 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2adb2ccc4e3e4c1396032eae34eeb18c" } modify_external_catalogs: true
I20251028 02:59:57.143246 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 583d6226a5f44d47a5ea4788c2b0d5ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.143271 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 0e19a2bdc7024b0c9fe84f8983f9d292
I20251028 02:59:57.143568 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 0e19a2bdc7024b0c9fe84f8983f9d292 deleted and replaced by tablet d37c267aaee7412b869753164556b9c0
I20251028 02:59:57.143680 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 72350c60d73b41829d99ac8d01aa04bc from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.144438 8552 ts_tablet_manager.cc:1929] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.144459 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 5155389485d54c1b82c45eb65364620f from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.145411 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35e211a1a8c547baaee1199f7103bfad" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:57.145910 8083 leader_election.cc:304] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
W20251028 02:59:57.146035 8083 raft_consensus.cc:2686] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251028 02:59:57.146322 9639 consensus_queue.cc:1048] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.143002 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6dd563e13e624d4bb88950b46d480876
I20251028 02:59:57.146744 7963 catalog_manager.cc:6719] ReplaceTablet: tablet 6dd563e13e624d4bb88950b46d480876 deleted and replaced by tablet 2fc2faa9c4f04fe3b931ce8a2808d738
I20251028 02:59:57.147397 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 with delete_type TABLET_DATA_DELETED (replaced by tablet d37c267aaee7412b869753164556b9c0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.147742 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 8483cbc3099644d2b6f7ff52869e21ea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.148898 9401 consensus_queue.cc:1048] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.147578 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4ff8c974231f4b8aa18c18a05ad3e738 (table default.Table_bfd3f08061c3433f878612643c04387b [id=a015c31e18c74191879fe1474dcb6ab2]) successfully deleted
W20251028 02:59:57.149294 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 5bf9795bd59b42ea997a97c84e1ea111 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bf9795bd59b42ea997a97c84e1ea111 already in progress: creating tablet
I20251028 02:59:57.150032 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dd563e13e624d4bb88950b46d480876 with delete_type TABLET_DATA_DELETED (replaced by tablet 2fc2faa9c4f04fe3b931ce8a2808d738) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.151079 8551 raft_consensus.cc:2243] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.151381 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 4e44345b09584bbca19b19e57ce17c77 with delete_type TABLET_DATA_DELETED (replaced by tablet c8b4b0aa30194ba495adca9138e2454e) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.152274 8545 tablet_bootstrap.cc:492] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:57.152976 7963 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 2bd4558fc315491687a12efce7ee8ab2 (table default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9]). Expected version 1 got 0
I20251028 02:59:57.153455 8545 tablet_bootstrap.cc:654] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.154155 8552 log.cc:1199] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/2b5908fd54944004866ca3864e8131d2
I20251028 02:59:57.154428 8552 ts_tablet_manager.cc:1950] T 2b5908fd54944004866ca3864e8131d2 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.147225 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 8483cbc3099644d2b6f7ff52869e21ea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.150388 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dd563e13e624d4bb88950b46d480876 with delete_type TABLET_DATA_DELETED (replaced by tablet 2fc2faa9c4f04fe3b931ce8a2808d738) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.159739 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 2bd4558fc315491687a12efce7ee8ab2 (table default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9]). Expected version 1 got 0
I20251028 02:59:57.160812 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 with delete_type TABLET_DATA_DELETED (replaced by tablet d37c267aaee7412b869753164556b9c0) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.162283 8552 tablet_replica.cc:333] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.162452 8552 raft_consensus.cc:2243] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.162551 8552 raft_consensus.cc:2272] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.162932 8552 ts_tablet_manager.cc:1916] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:57.164999 7953 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.154.189:45473, user_credentials={real_user=slave}} blocked reactor thread for 91663.8us
I20251028 02:59:57.165263 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 72350c60d73b41829d99ac8d01aa04bc
I20251028 02:59:57.165354 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3435803c6167447a8b3afba6c0f3511b
I20251028 02:59:57.165524 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 72350c60d73b41829d99ac8d01aa04bc deleted and replaced by tablet 39f0a9883b584083b542d078ea7c165b
I20251028 02:59:57.166620 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 597d792ed13941a2a895b0dc305614f1
I20251028 02:59:57.166934 8552 ts_tablet_manager.cc:1929] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.166934 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 2893b3a6e8854187b837a7e6fef58c4e with delete_type TABLET_DATA_DELETED (replaced by tablet f5e7e0614af04cc2a22130906d062a79) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.167021 8552 log.cc:1199] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/551aea93ba064bc89ae00b002eb7ccfc
I20251028 02:59:57.167137 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet 8483cbc3099644d2b6f7ff52869e21ea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.167239 8552 ts_tablet_manager.cc:1950] T 551aea93ba064bc89ae00b002eb7ccfc P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.168303 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 with delete_type TABLET_DATA_DELETED (replaced by tablet d37c267aaee7412b869753164556b9c0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.168767 8552 tablet_replica.cc:333] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.168916 8552 raft_consensus.cc:2243] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.169021 8552 raft_consensus.cc:2272] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.169601 8552 ts_tablet_manager.cc:1916] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.169958 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4e44345b09584bbca19b19e57ce17c77 (table default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]) successfully deleted
I20251028 02:59:57.170079 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dd563e13e624d4bb88950b46d480876 with delete_type TABLET_DATA_DELETED (replaced by tablet 2fc2faa9c4f04fe3b931ce8a2808d738) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.170130 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9e260030c0e6423cba37a2e3af1c6400
I20251028 02:59:57.170398 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 72350c60d73b41829d99ac8d01aa04bc with delete_type TABLET_DATA_DELETED (replaced by tablet 39f0a9883b584083b542d078ea7c165b) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.170576 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 3435803c6167447a8b3afba6c0f3511b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.170949 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 597d792ed13941a2a895b0dc305614f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.171725 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e260030c0e6423cba37a2e3af1c6400 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.171754 8397 tablet_service.cc:1505] Processing CreateTablet for tablet d60fc654f6af44b290d46235082a4113 (DEFAULT_TABLE table=default.Table_dbd811be59344e8587fff44779930d85 [id=94895dadf0e34a19a454e8755c61c785]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.172060 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d60fc654f6af44b290d46235082a4113. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.172901 8552 ts_tablet_manager.cc:1929] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.173014 8552 log.cc:1199] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/35e211a1a8c547baaee1199f7103bfad
I20251028 02:59:57.173310 8552 ts_tablet_manager.cc:1950] T 35e211a1a8c547baaee1199f7103bfad P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.173308 7965 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet a808098c70b54ad3a0230352af0b743c (table default.Table_9e950f6a07524cb0b5a0d3d967668ad6 [id=5dd4cb94fa384facbdd1aef677a4b498]). Expected version 1 got 0
I20251028 02:59:57.173414 7965 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 2bd4558fc315491687a12efce7ee8ab2 (table default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9]). Expected version 1 got 0
I20251028 02:59:57.179172 8552 tablet_replica.cc:333] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.179379 8552 raft_consensus.cc:2243] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.179559 8552 raft_consensus.cc:2272] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.180023 8552 ts_tablet_manager.cc:1916] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.180124 8544 tablet_bootstrap.cc:492] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.180248 8544 ts_tablet_manager.cc:1403] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.060s user 0.002s sys 0.001s
I20251028 02:59:57.180799 8544 raft_consensus.cc:359] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.180912 8544 raft_consensus.cc:385] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.180944 8544 raft_consensus.cc:740] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.181066 8544 consensus_queue.cc:260] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.181368 8544 ts_tablet_manager.cc:1434] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.181967 8544 tablet_bootstrap.cc:492] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.182188 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet 72350c60d73b41829d99ac8d01aa04bc with delete_type TABLET_DATA_DELETED (replaced by tablet 39f0a9883b584083b542d078ea7c165b) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.182600 8552 ts_tablet_manager.cc:1929] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.182680 8552 log.cc:1199] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/929cf41ac7b0437387fbc488983f5cac
I20251028 02:59:57.182906 8552 ts_tablet_manager.cc:1950] T 929cf41ac7b0437387fbc488983f5cac P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.183058 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet 3435803c6167447a8b3afba6c0f3511b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.183177 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet 597d792ed13941a2a895b0dc305614f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.183324 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 597d792ed13941a2a895b0dc305614f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.184695 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e260030c0e6423cba37a2e3af1c6400 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.186090 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e260030c0e6423cba37a2e3af1c6400 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.186841 9595 consensus_queue.cc:1048] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.187254 9551 tablet.cc:1722] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:57.188097 9526 tablet.cc:1722] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:57.189906 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 2b5908fd54944004866ca3864e8131d2 (table default.Table_2cd00f3aa8f744bd890e7fc0efb34753 [id=0ff7a9d5241d4633a3172368b0efddb6]) successfully deleted
W20251028 02:59:57.190068 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 626c2cc2d19047b794bf16c1b078941f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 626c2cc2d19047b794bf16c1b078941f already in progress: creating tablet
W20251028 02:59:57.190197 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 6599ec60c3004e349e7e576e4d13c4fe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6599ec60c3004e349e7e576e4d13c4fe already in progress: creating tablet
W20251028 02:59:57.190315 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 3f06af61617e4be9a753692e9ad4978d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f06af61617e4be9a753692e9ad4978d already in progress: creating tablet
I20251028 02:59:57.190465 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 551aea93ba064bc89ae00b002eb7ccfc (table default.Table_806d85836a4c44a986095f5108a93d7f [id=c098c2727dd147e69471353bd9a741fc]) successfully deleted
W20251028 02:59:57.190593 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 6599ec60c3004e349e7e576e4d13c4fe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6599ec60c3004e349e7e576e4d13c4fe already in progress: creating tablet
I20251028 02:59:57.191017 8552 tablet_replica.cc:333] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.191206 8552 raft_consensus.cc:2243] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.193480 8544 tablet_bootstrap.cc:654] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.194373 8144 raft_consensus.cc:1275] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:57.194597 9529 tablet.cc:1722] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20251028 02:59:57.195086 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a19eeac727e142579e34a301a6e2db02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a19eeac727e142579e34a301a6e2db02 already in progress: creating tablet
W20251028 02:59:57.195245 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet f2d0dffdf5cc414ab870e0dde75916d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2d0dffdf5cc414ab870e0dde75916d6 already in progress: creating tablet
I20251028 02:59:57.195350 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 35e211a1a8c547baaee1199f7103bfad (table default.Table_26410f48ded64d1285940aeff4f1fc07 [id=3191deea26744b009e341282c1babcbf]) successfully deleted
I20251028 02:59:57.195438 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 929cf41ac7b0437387fbc488983f5cac (table default.Table_aa25a85ac8d24940a727a586d6c630db [id=10f40fa1652f495e91fcbd5852bc3464]) successfully deleted
I20251028 02:59:57.195417 9596 consensus_queue.cc:1048] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:57.195531 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 9e4eec90a2fa412bb0954805f5b8d832 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9e4eec90a2fa412bb0954805f5b8d832 already in progress: creating tablet
I20251028 02:59:57.196323 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 3a5415a4246c47ca896751f2fd68af71 (DEFAULT_TABLE table=default.Table_316b63d9aec64a06b194dca118966b47 [id=077d81b3977448b2ae212436d5014339]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.196631 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a5415a4246c47ca896751f2fd68af71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.196810 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 463f1c3ac2d74791830431191601282c (DEFAULT_TABLE table=default.Table_831a31c4a8fe414e8c9e8462b95a612e [id=e04ba046a39944a78de902cd6377c58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.197021 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463f1c3ac2d74791830431191601282c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.198899 8396 tablet_service.cc:1505] Processing CreateTablet for tablet f1d727c3dbef467da0aec436ffc393be (DEFAULT_TABLE table=default.Table_fcb100729f5b48b990a9c4d2d9b9fcf2 [id=0381bc31864343c19a461fe49735d58a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.198938 9639 consensus_queue.cc:1048] T bf7f6a7ada824eb1bca9be5344c9287e P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.199162 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d727c3dbef467da0aec436ffc393be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.200984 8551 raft_consensus.cc:2272] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.201346 8251 tablet_service.cc:1505] Processing CreateTablet for tablet f1d727c3dbef467da0aec436ffc393be (DEFAULT_TABLE table=default.Table_fcb100729f5b48b990a9c4d2d9b9fcf2 [id=0381bc31864343c19a461fe49735d58a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.201627 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d727c3dbef467da0aec436ffc393be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.201716 8551 ts_tablet_manager.cc:1916] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.204718 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_19f5792cad7f4b2fa7f0446f6c3c7bb0 [id=da79674aa8cd468e84114457ee46766e] alter complete (version 2)
I20251028 02:59:57.204826 8562 raft_consensus.cc:493] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.204917 8562 raft_consensus.cc:515] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.205335 8562 leader_election.cc:290] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.205928 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9583046f549d411a9e341560f428381c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.198899 8384 tablet_service.cc:1505] Processing CreateTablet for tablet 59cd8e6357fe4a6cae6be04c493a6880 (DEFAULT_TABLE table=default.Table_d5216511b6cb4223afa3920eec378c6b [id=9a604bc56e0640f38093ad6d9d04930c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.206225 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9583046f549d411a9e341560f428381c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.206271 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59cd8e6357fe4a6cae6be04c493a6880. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.207693 8544 tablet_bootstrap.cc:492] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.207823 8544 ts_tablet_manager.cc:1403] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.001s
W20251028 02:59:57.208635 8083 leader_election.cc:343] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.208721 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet fef84b69b1b746d4b4feeed96670c67b (table default.Table_2f27286185dc41e4b5bc773a4ccdd68a [id=cf715387170e40c992807874e34c66b7]). Expected version 1 got 0
I20251028 02:59:57.208676 8544 raft_consensus.cc:359] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:57.208770 8083 leader_election.cc:343] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.208797 8544 raft_consensus.cc:385] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.208811 8083 leader_election.cc:304] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.208851 8544 raft_consensus.cc:740] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.208971 8544 consensus_queue.cc:260] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.209158 9650 raft_consensus.cc:2749] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.209445 8544 ts_tablet_manager.cc:1434] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:57.209633 8544 tablet_bootstrap.cc:492] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.209642 8552 raft_consensus.cc:2272] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.211526 8544 tablet_bootstrap.cc:654] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.212332 8551 ts_tablet_manager.cc:1929] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.213289 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet fef84b69b1b746d4b4feeed96670c67b (table default.Table_2f27286185dc41e4b5bc773a4ccdd68a [id=cf715387170e40c992807874e34c66b7]). Expected version 1 got 0
I20251028 02:59:57.214367 8552 ts_tablet_manager.cc:1916] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.214664 9551 tablet.cc:1722] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.217295 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 463f1c3ac2d74791830431191601282c (DEFAULT_TABLE table=default.Table_831a31c4a8fe414e8c9e8462b95a612e [id=e04ba046a39944a78de902cd6377c58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.217584 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463f1c3ac2d74791830431191601282c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.218680 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet fef84b69b1b746d4b4feeed96670c67b (table default.Table_2f27286185dc41e4b5bc773a4ccdd68a [id=cf715387170e40c992807874e34c66b7]). Expected version 1 got 0
I20251028 02:59:57.219559 8552 ts_tablet_manager.cc:1929] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.219751 8552 log.cc:1199] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/26a344e2e03e4eeaa688275e9ecbf63f
I20251028 02:59:57.220108 8552 ts_tablet_manager.cc:1950] T 26a344e2e03e4eeaa688275e9ecbf63f P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.221832 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 26a344e2e03e4eeaa688275e9ecbf63f (table default.Table_152433eb6e11487198b81047f13cc1f1 [id=8642a2095b8049de90e6df3a8570d730]) successfully deleted
I20251028 02:59:57.223246 8551 log.cc:1199] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/35e211a1a8c547baaee1199f7103bfad
I20251028 02:59:57.215610 9526 tablet.cc:1722] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.218107 9639 consensus_queue.cc:1048] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.225419 8551 ts_tablet_manager.cc:1950] T 35e211a1a8c547baaee1199f7103bfad P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.219756 9646 raft_consensus.cc:493] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.227730 9646 raft_consensus.cc:515] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.228192 9646 leader_election.cc:290] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:57.222195 8083 consensus_peers.cc:597] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 551aea93ba064bc89ae00b002eb7ccfc. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:57.228797 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5428db004de74f7492c9b8999e53020e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.228965 9646 raft_consensus.cc:493] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.229051 9646 raft_consensus.cc:515] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:57.229310 8083 leader_election.cc:343] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.229372 9646 leader_election.cc:290] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.229604 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5428db004de74f7492c9b8999e53020e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.230038 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 59cd8e6357fe4a6cae6be04c493a6880 (DEFAULT_TABLE table=default.Table_d5216511b6cb4223afa3920eec378c6b [id=9a604bc56e0640f38093ad6d9d04930c]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:57.230271 8083 leader_election.cc:343] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.230336 8083 leader_election.cc:304] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.230319 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "529b865389f94471a6849d17ff206e4c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.230525 9646 raft_consensus.cc:2749] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251028 02:59:57.231117 8083 leader_election.cc:343] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.231132 7965 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_2f27286185dc41e4b5bc773a4ccdd68a [id=cf715387170e40c992807874e34c66b7] alter complete (version 1)
I20251028 02:59:57.231525 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 35e211a1a8c547baaee1199f7103bfad (table default.Table_26410f48ded64d1285940aeff4f1fc07 [id=3191deea26744b009e341282c1babcbf]) successfully deleted
I20251028 02:59:57.231534 9639 consensus_queue.cc:1048] T c1470187cdc74716b727052ea2b0b4a0 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:57.231698 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 9e4eec90a2fa412bb0954805f5b8d832 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9e4eec90a2fa412bb0954805f5b8d832 already in progress: creating tablet
I20251028 02:59:57.232061 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "529b865389f94471a6849d17ff206e4c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.232436 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59cd8e6357fe4a6cae6be04c493a6880. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.233515 8551 tablet_replica.cc:333] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.233721 8551 raft_consensus.cc:2243] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
W20251028 02:59:57.233716 8083 leader_election.cc:343] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.233793 8083 leader_election.cc:304] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.233991 9646 raft_consensus.cc:2749] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.234184 8551 raft_consensus.cc:2272] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.234813 8551 ts_tablet_manager.cc:1916] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:57.234172 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a76e16d10a174adda26b8ddd49ee528c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a76e16d10a174adda26b8ddd49ee528c already in progress: creating tablet
W20251028 02:59:57.235250 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a76e16d10a174adda26b8ddd49ee528c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a76e16d10a174adda26b8ddd49ee528c already in progress: creating tablet
W20251028 02:59:57.235491 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 1c41d49f9eba40a48bd6900671297db9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c41d49f9eba40a48bd6900671297db9 already in progress: creating tablet
W20251028 02:59:57.235845 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet f6ac7b5fff0848cebf710ba3ed50def4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6ac7b5fff0848cebf710ba3ed50def4 already in progress: creating tablet
I20251028 02:59:57.235906 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8b7624342986474fbd0e31c86947f781" } modify_external_catalogs: true
I20251028 02:59:57.236093 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8b7624342986474fbd0e31c86947f781" } modify_external_catalogs: true
I20251028 02:59:57.236538 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 3a5415a4246c47ca896751f2fd68af71 (DEFAULT_TABLE table=default.Table_316b63d9aec64a06b194dca118966b47 [id=077d81b3977448b2ae212436d5014339]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:57.236701 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 45572222af854b34a5f26a00f8ad9615 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45572222af854b34a5f26a00f8ad9615 already in progress: creating tablet
I20251028 02:59:57.236907 8552 tablet_replica.cc:333] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.237010 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a5415a4246c47ca896751f2fd68af71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.237102 8552 raft_consensus.cc:2243] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.237250 8552 raft_consensus.cc:2272] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.237737 8552 ts_tablet_manager.cc:1916] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.240741 8551 ts_tablet_manager.cc:1929] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.240824 8551 log.cc:1199] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/26a344e2e03e4eeaa688275e9ecbf63f
I20251028 02:59:57.241065 8551 ts_tablet_manager.cc:1950] T 26a344e2e03e4eeaa688275e9ecbf63f P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.241703 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 26a344e2e03e4eeaa688275e9ecbf63f (table default.Table_152433eb6e11487198b81047f13cc1f1 [id=8642a2095b8049de90e6df3a8570d730]) successfully deleted
I20251028 02:59:57.241971 8552 ts_tablet_manager.cc:1929] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.242303 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f64a73f0705a4a119870d8d17497b338" } new_table_name: "default.Table_b2e295519dda4780ba93dede43e9b41c" modify_external_catalogs: true
I20251028 02:59:57.246384 8552 log.cc:1199] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/49b0b3459d434cf2810029ce205e745c
I20251028 02:59:57.246762 8552 ts_tablet_manager.cc:1950] T 49b0b3459d434cf2810029ce205e745c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.247583 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 49b0b3459d434cf2810029ce205e745c (table default.Table_75e654d01f4147b3b7d928c99967c5cc [id=f9132fe51a804151b8df8e1e8fc0d220]) successfully deleted
W20251028 02:59:57.248131 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 3f04c867f0ce4ab99f980cca063abc30 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f04c867f0ce4ab99f980cca063abc30 already in progress: creating tablet
W20251028 02:59:57.248587 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 9621e42e6a324ecda50fe83c0c53119c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9621e42e6a324ecda50fe83c0c53119c already in progress: creating tablet
W20251028 02:59:57.249061 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a9653fc3236f4e0b812de5a859932587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9653fc3236f4e0b812de5a859932587 already in progress: creating tablet
W20251028 02:59:57.249478 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 0fcabc4a2fd44043be5451046ff5bd97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fcabc4a2fd44043be5451046ff5bd97 already in progress: creating tablet
I20251028 02:59:57.249511 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 72350c60d73b41829d99ac8d01aa04bc with delete_type TABLET_DATA_DELETED (replaced by tablet 39f0a9883b584083b542d078ea7c165b) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:57.251082 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 39f0a9883b584083b542d078ea7c165b has no consensus state
I20251028 02:59:57.252146 8552 tablet_replica.cc:333] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.252385 8552 raft_consensus.cc:2243] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.252583 8552 raft_consensus.cc:2272] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.253058 8552 ts_tablet_manager.cc:1916] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.255671 8552 ts_tablet_manager.cc:1929] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.6
I20251028 02:59:57.255805 8552 log.cc:1199] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9e908d5921d34c9396c1ccdc857c66b7
I20251028 02:59:57.256191 8552 ts_tablet_manager.cc:1950] T 9e908d5921d34c9396c1ccdc857c66b7 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.257050 8552 tablet_replica.cc:333] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.257252 8552 raft_consensus.cc:2243] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.256990 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9e908d5921d34c9396c1ccdc857c66b7 (table default.Table_8f6fdbd222f4456596107350e6c3c6e1 [id=4eb2743632814baab2e39bbdce700d6b]) successfully deleted
I20251028 02:59:57.257424 8552 raft_consensus.cc:2272] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:57.257781 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet bd50e7d4e28541de9799833c53fe7c02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd50e7d4e28541de9799833c53fe7c02 already in progress: creating tablet
W20251028 02:59:57.257963 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 28b81f0000c4407096feff773c29579d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28b81f0000c4407096feff773c29579d already in progress: creating tablet
I20251028 02:59:57.258278 8552 ts_tablet_manager.cc:1916] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:57.260010 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 1c41d49f9eba40a48bd6900671297db9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c41d49f9eba40a48bd6900671297db9 already in progress: creating tablet
W20251028 02:59:57.260809 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f6ac7b5fff0848cebf710ba3ed50def4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6ac7b5fff0848cebf710ba3ed50def4 already in progress: creating tablet
I20251028 02:59:57.260869 8544 tablet_bootstrap.cc:492] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.260963 8544 ts_tablet_manager.cc:1403] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.051s user 0.003s sys 0.000s
W20251028 02:59:57.261245 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 45572222af854b34a5f26a00f8ad9615 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45572222af854b34a5f26a00f8ad9615 already in progress: creating tablet
I20251028 02:59:57.261423 8551 tablet_replica.cc:333] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.261515 8544 raft_consensus.cc:359] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.261571 8551 raft_consensus.cc:2243] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.261616 8544 raft_consensus.cc:385] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.261646 8544 raft_consensus.cc:740] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.261679 8551 raft_consensus.cc:2272] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.261773 8544 consensus_queue.cc:260] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.262300 8551 ts_tablet_manager.cc:1916] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.263015 8552 ts_tablet_manager.cc:1929] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.263127 8552 log.cc:1199] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4e44345b09584bbca19b19e57ce17c77
I20251028 02:59:57.263427 8552 ts_tablet_manager.cc:1950] T 4e44345b09584bbca19b19e57ce17c77 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.264160 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4e44345b09584bbca19b19e57ce17c77 (table default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]) successfully deleted
I20251028 02:59:57.264437 8552 tablet_replica.cc:333] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.264575 8551 ts_tablet_manager.cc:1929] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.264659 8551 log.cc:1199] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/551aea93ba064bc89ae00b002eb7ccfc
I20251028 02:59:57.264617 8552 raft_consensus.cc:2243] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.264914 8552 raft_consensus.cc:2272] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.264940 8551 ts_tablet_manager.cc:1950] T 551aea93ba064bc89ae00b002eb7ccfc P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.265838 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 551aea93ba064bc89ae00b002eb7ccfc (table default.Table_806d85836a4c44a986095f5108a93d7f [id=c098c2727dd147e69471353bd9a741fc]) successfully deleted
I20251028 02:59:57.266611 8544 ts_tablet_manager.cc:1434] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251028 02:59:57.266886 8544 tablet_bootstrap.cc:492] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.268329 8544 tablet_bootstrap.cc:654] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.269280 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 4393587f022446828e344a9c394a92f9 (DEFAULT_TABLE table=default.Table_0ce46dd2a59848399a1d4fdeac7deef9 [id=e574f40b6ddb4eccae167d16875b0e7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.269698 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4393587f022446828e344a9c394a92f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.272233 8551 tablet_replica.cc:333] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.272459 8551 raft_consensus.cc:2243] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.272703 8551 raft_consensus.cc:2272] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.273238 8551 ts_tablet_manager.cc:1916] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:57.274298 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 3f04c867f0ce4ab99f980cca063abc30 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f04c867f0ce4ab99f980cca063abc30 already in progress: creating tablet
W20251028 02:59:57.274482 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 9621e42e6a324ecda50fe83c0c53119c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9621e42e6a324ecda50fe83c0c53119c already in progress: creating tablet
W20251028 02:59:57.274605 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a9653fc3236f4e0b812de5a859932587 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9653fc3236f4e0b812de5a859932587 already in progress: creating tablet
W20251028 02:59:57.274717 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet bd50e7d4e28541de9799833c53fe7c02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd50e7d4e28541de9799833c53fe7c02 already in progress: creating tablet
I20251028 02:59:57.275555 8400 tablet_service.cc:1505] Processing CreateTablet for tablet d195958488bf401c8668116a97bf0f58 (DEFAULT_TABLE table=default.Table_62c4ca6858344881a02383c7919d85cb [id=d95696d7d42c4d999f99b584a7232245]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.275637 8389 tablet_service.cc:1505] Processing CreateTablet for tablet 2fc2faa9c4f04fe3b931ce8a2808d738 (DEFAULT_TABLE table=default.Table_ee1375b2eca6432291e041e9aaf7bf3a [id=eaebbf79fab746dfaa81407d3509306c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.275703 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 5fa32a1433cd47a9ad39d1c613a18761 (DEFAULT_TABLE table=default.Table_bc8cc7f063644f3eaa9fb0abce91091a [id=bae2c0c5846d4cf69eeac13da536c4cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.275821 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d195958488bf401c8668116a97bf0f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.275957 8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fc2faa9c4f04fe3b931ce8a2808d738. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.276214 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 0ee59aeedd2c4c9ba94afe6578af649a (DEFAULT_TABLE table=default.Table_033167c60f644376b85c64d13503579f [id=7615c3654d9746ca87d81bdacb360935]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.276353 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 39f0a9883b584083b542d078ea7c165b (DEFAULT_TABLE table=default.Table_b2e295519dda4780ba93dede43e9b41c [id=5de7688626854f0c8cabe8fb32b28ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.276445 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee59aeedd2c4c9ba94afe6578af649a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.276553 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39f0a9883b584083b542d078ea7c165b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.276654 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 5fa32a1433cd47a9ad39d1c613a18761 (DEFAULT_TABLE table=default.Table_bc8cc7f063644f3eaa9fb0abce91091a [id=bae2c0c5846d4cf69eeac13da536c4cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.276933 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa32a1433cd47a9ad39d1c613a18761. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.278932 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 2fc2faa9c4f04fe3b931ce8a2808d738 (DEFAULT_TABLE table=default.Table_ee1375b2eca6432291e041e9aaf7bf3a [id=eaebbf79fab746dfaa81407d3509306c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.279206 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fc2faa9c4f04fe3b931ce8a2808d738. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.279583 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 39f0a9883b584083b542d078ea7c165b (DEFAULT_TABLE table=default.Table_b2e295519dda4780ba93dede43e9b41c [id=5de7688626854f0c8cabe8fb32b28ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.279803 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39f0a9883b584083b542d078ea7c165b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.281303 8256 tablet_service.cc:1505] Processing CreateTablet for tablet d195958488bf401c8668116a97bf0f58 (DEFAULT_TABLE table=default.Table_62c4ca6858344881a02383c7919d85cb [id=d95696d7d42c4d999f99b584a7232245]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.281527 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d195958488bf401c8668116a97bf0f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.282081 8551 ts_tablet_manager.cc:1929] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.6
I20251028 02:59:57.282063 8253 tablet_service.cc:1505] Processing CreateTablet for tablet 0ee59aeedd2c4c9ba94afe6578af649a (DEFAULT_TABLE table=default.Table_033167c60f644376b85c64d13503579f [id=7615c3654d9746ca87d81bdacb360935]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.282156 8551 log.cc:1199] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9e908d5921d34c9396c1ccdc857c66b7
I20251028 02:59:57.282265 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee59aeedd2c4c9ba94afe6578af649a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.282410 8551 ts_tablet_manager.cc:1950] T 9e908d5921d34c9396c1ccdc857c66b7 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.283028 8551 tablet_replica.cc:333] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.283176 8551 raft_consensus.cc:2243] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.283360 8551 pending_rounds.cc:70] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:57.283447 8551 pending_rounds.cc:77] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597146444898304 op_type: NO_OP noop_request { }
I20251028 02:59:57.283522 8551 raft_consensus.cc:2889] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:57.283602 8551 raft_consensus.cc:2272] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.284212 8551 ts_tablet_manager.cc:1916] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.284298 9646 raft_consensus.cc:493] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.284534 9646 raft_consensus.cc:515] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.284739 8386 tablet_service.cc:1505] Processing CreateTablet for tablet d37c267aaee7412b869753164556b9c0 (DEFAULT_TABLE table=default.Table_4f7a6b08bb2b4cee90e54afd8f1b801a [id=243b57c791df41819156363e8607623d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.284986 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37c267aaee7412b869753164556b9c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.285025 9646 leader_election.cc:290] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.285159 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa32a1433cd47a9ad39d1c613a18761. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.285485 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 5d5c68c16c3e4d37be7f9a505ab44549 (DEFAULT_TABLE table=default.Table_8249c2c329814e99a3636633a900f8bd [id=b441e3de2e844b77a8717ecb2a48809a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.285697 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d5c68c16c3e4d37be7f9a505ab44549. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.286168 8551 ts_tablet_manager.cc:1929] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.286217 8551 log.cc:1199] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/49b0b3459d434cf2810029ce205e745c
I20251028 02:59:57.286397 8551 ts_tablet_manager.cc:1950] T 49b0b3459d434cf2810029ce205e745c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.287144 8551 tablet_replica.cc:333] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.287266 8551 raft_consensus.cc:2243] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.287356 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13de247dfb18478daac9ca5601220faa" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.287433 8551 raft_consensus.cc:2272] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:57.287835 8083 leader_election.cc:343] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.288496 8551 ts_tablet_manager.cc:1916] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.290230 8251 tablet_service.cc:1505] Processing CreateTablet for tablet d37c267aaee7412b869753164556b9c0 (DEFAULT_TABLE table=default.Table_4f7a6b08bb2b4cee90e54afd8f1b801a [id=243b57c791df41819156363e8607623d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.290539 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9e908d5921d34c9396c1ccdc857c66b7 (table default.Table_8f6fdbd222f4456596107350e6c3c6e1 [id=4eb2743632814baab2e39bbdce700d6b]) successfully deleted
W20251028 02:59:57.290694 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 0fcabc4a2fd44043be5451046ff5bd97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fcabc4a2fd44043be5451046ff5bd97 already in progress: creating tablet
I20251028 02:59:57.290683 8384 tablet_service.cc:1505] Processing CreateTablet for tablet c20410feb13543db9642c1e40bfb9300 (DEFAULT_TABLE table=default.Table_e0b6845d183a4c93b6f02c99691cac6d [id=9d476ef5a23a41409955b050390ab263]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.290787 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 49b0b3459d434cf2810029ce205e745c (table default.Table_75e654d01f4147b3b7d928c99967c5cc [id=f9132fe51a804151b8df8e1e8fc0d220]) successfully deleted
W20251028 02:59:57.290901 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 28b81f0000c4407096feff773c29579d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28b81f0000c4407096feff773c29579d already in progress: creating tablet
W20251028 02:59:57.291003 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 2893b3a6e8854187b837a7e6fef58c4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2893b3a6e8854187b837a7e6fef58c4e already in progress: creating tablet
W20251028 02:59:57.291100 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e19a2bdc7024b0c9fe84f8983f9d292 already in progress: creating tablet
W20251028 02:59:57.291208 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 8483cbc3099644d2b6f7ff52869e21ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8483cbc3099644d2b6f7ff52869e21ea already in progress: creating tablet
W20251028 02:59:57.291314 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 6dd563e13e624d4bb88950b46d480876 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6dd563e13e624d4bb88950b46d480876 already in progress: creating tablet
I20251028 02:59:57.291477 8551 ts_tablet_manager.cc:1929] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.291539 8551 log.cc:1199] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4e44345b09584bbca19b19e57ce17c77
I20251028 02:59:57.291632 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20410feb13543db9642c1e40bfb9300. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.291757 8551 ts_tablet_manager.cc:1950] T 4e44345b09584bbca19b19e57ce17c77 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.292356 8551 tablet_replica.cc:333] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.292362 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_272a84b9ac3e46c882522d75055385d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.292413 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13de247dfb18478daac9ca5601220faa" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.292594 8551 raft_consensus.cc:2243] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.292578 8271 raft_consensus.cc:2468] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:57.292717 8551 raft_consensus.cc:2272] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:57.292826 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_272a84b9ac3e46c882522d75055385d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.292965 8083 leader_election.cc:304] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.293273 9646 raft_consensus.cc:2804] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:57.293322 9646 raft_consensus.cc:493] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:57.293357 9646 raft_consensus.cc:3060] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.294389 9646 raft_consensus.cc:515] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.294734 9646 leader_election.cc:290] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.295367 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37c267aaee7412b869753164556b9c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.295621 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13de247dfb18478daac9ca5601220faa" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:57.295881 8551 ts_tablet_manager.cc:1916] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.297122 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ba64524142934af08658e434e7fb08ed
I20251028 02:59:57.297744 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13de247dfb18478daac9ca5601220faa" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:57.297863 8271 raft_consensus.cc:3060] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:57.298009 8083 leader_election.cc:343] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.298082 8551 ts_tablet_manager.cc:1929] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.298144 8551 log.cc:1199] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/72350c60d73b41829d99ac8d01aa04bc
I20251028 02:59:57.298372 8551 ts_tablet_manager.cc:1950] T 72350c60d73b41829d99ac8d01aa04bc P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.298827 8271 raft_consensus.cc:2468] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:57.299098 8385 tablet_service.cc:1505] Processing CreateTablet for tablet f5e7e0614af04cc2a22130906d062a79 (DEFAULT_TABLE table=default.Table_85b63c813dc94719b07f0e12854b164b [id=49bac53017824831928c69211121863b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.299391 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5e7e0614af04cc2a22130906d062a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.299711 8083 leader_election.cc:304] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.299940 9650 raft_consensus.cc:2804] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:57.300014 9650 raft_consensus.cc:697] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:57.300158 9650 consensus_queue.cc:237] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.301975 8551 tablet_replica.cc:333] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.302174 8551 raft_consensus.cc:2243] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.302335 8551 raft_consensus.cc:2272] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.303633 8143 raft_consensus.cc:1275] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:57.304328 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4e44345b09584bbca19b19e57ce17c77 (table default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]) successfully deleted
W20251028 02:59:57.304368 8220 consensus_peers.cc:597] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:57.304325 9639 consensus_queue.cc:1048] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.304541 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 72350c60d73b41829d99ac8d01aa04bc (table default.Table_b2e295519dda4780ba93dede43e9b41c [id=5de7688626854f0c8cabe8fb32b28ede]) successfully deleted
W20251028 02:59:57.304737 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 597d792ed13941a2a895b0dc305614f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 597d792ed13941a2a895b0dc305614f1 already in progress: creating tablet
I20251028 02:59:57.304917 8551 ts_tablet_manager.cc:1916] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.305224 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet ba64524142934af08658e434e7fb08ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.305610 8263 tablet_service.cc:1505] Processing CreateTablet for tablet c20410feb13543db9642c1e40bfb9300 (DEFAULT_TABLE table=default.Table_e0b6845d183a4c93b6f02c99691cac6d [id=9d476ef5a23a41409955b050390ab263]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.305917 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20410feb13543db9642c1e40bfb9300. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.306051 8552 ts_tablet_manager.cc:1916] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.306313 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 5d5c68c16c3e4d37be7f9a505ab44549 (DEFAULT_TABLE table=default.Table_8249c2c329814e99a3636633a900f8bd [id=b441e3de2e844b77a8717ecb2a48809a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.306571 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d5c68c16c3e4d37be7f9a505ab44549. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.306758 8271 raft_consensus.cc:1275] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:57.307565 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet ba64524142934af08658e434e7fb08ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.307727 8551 ts_tablet_manager.cc:1929] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.307705 9639 consensus_queue.cc:1048] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.308321 8551 log.cc:1199] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9e260030c0e6423cba37a2e3af1c6400
I20251028 02:59:57.308636 8551 ts_tablet_manager.cc:1950] T 9e260030c0e6423cba37a2e3af1c6400 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.309386 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9e260030c0e6423cba37a2e3af1c6400 (table default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]) successfully deleted
W20251028 02:59:57.309903 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ba64524142934af08658e434e7fb08ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba64524142934af08658e434e7fb08ed already in progress: creating tablet
I20251028 02:59:57.310875 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 06302d2cb5554805b85916741ea10f0c (table default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d]). Expected version 2 got 0
I20251028 02:59:57.312408 9646 raft_consensus.cc:493] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.312506 9646 raft_consensus.cc:515] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.312852 9646 leader_election.cc:290] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.313293 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a808098c70b54ad3a0230352af0b743c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.313305 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a808098c70b54ad3a0230352af0b743c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:57.313746 8083 leader_election.cc:343] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:57.313858 8083 leader_election.cc:343] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.313911 8083 leader_election.cc:304] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.314126 9646 raft_consensus.cc:2749] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.316474 8552 ts_tablet_manager.cc:1929] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.316620 8552 log.cc:1199] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/04d271c95c2847ca8b1fd2d55741117c
I20251028 02:59:57.316946 8552 ts_tablet_manager.cc:1950] T 04d271c95c2847ca8b1fd2d55741117c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.318727 8552 tablet_replica.cc:333] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.319707 9526 tablet.cc:1722] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:57.315223 8259 tablet_service.cc:1505] Processing CreateTablet for tablet f5e7e0614af04cc2a22130906d062a79 (DEFAULT_TABLE table=default.Table_85b63c813dc94719b07f0e12854b164b [id=49bac53017824831928c69211121863b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.320132 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 4393587f022446828e344a9c394a92f9 (DEFAULT_TABLE table=default.Table_0ce46dd2a59848399a1d4fdeac7deef9 [id=e574f40b6ddb4eccae167d16875b0e7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.323504 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5e7e0614af04cc2a22130906d062a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.323225 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 04d271c95c2847ca8b1fd2d55741117c (table default.Table_ad3466e031ae40059b8e3ec01c7ed5a9 [id=f00f5e8a1fef48e480906cbf79121e33]) successfully deleted
W20251028 02:59:57.323640 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 2893b3a6e8854187b837a7e6fef58c4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2893b3a6e8854187b837a7e6fef58c4e already in progress: creating tablet
W20251028 02:59:57.323770 7955 catalog_manager.cc:4952] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 0703e18d7dba4d628e3cc310be55fcf5 because the tablet was not found. No further retry: Not found: Tablet not found: 0703e18d7dba4d628e3cc310be55fcf5
W20251028 02:59:57.323890 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 8483cbc3099644d2b6f7ff52869e21ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8483cbc3099644d2b6f7ff52869e21ea already in progress: creating tablet
I20251028 02:59:57.326905 9529 tablet.cc:1722] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:57.328182 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4393587f022446828e344a9c394a92f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.332916 8552 raft_consensus.cc:2243] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.333155 8552 raft_consensus.cc:2272] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.334473 7964 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d] alter complete (version 2)
I20251028 02:59:57.335933 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 9583046f549d411a9e341560f428381c with delete_type TABLET_DATA_DELETED (replaced by tablet ba64524142934af08658e434e7fb08ed) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.336295 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 59cd8e6357fe4a6cae6be04c493a6880 (DEFAULT_TABLE table=default.Table_d5216511b6cb4223afa3920eec378c6b [id=9a604bc56e0640f38093ad6d9d04930c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.336421 8116 tablet_service.cc:1505] Processing CreateTablet for tablet c20410feb13543db9642c1e40bfb9300 (DEFAULT_TABLE table=default.Table_e0b6845d183a4c93b6f02c99691cac6d [id=9d476ef5a23a41409955b050390ab263]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.336580 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59cd8e6357fe4a6cae6be04c493a6880. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.336738 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 5d5c68c16c3e4d37be7f9a505ab44549 (DEFAULT_TABLE table=default.Table_8249c2c329814e99a3636633a900f8bd [id=b441e3de2e844b77a8717ecb2a48809a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.336845 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 3a5415a4246c47ca896751f2fd68af71 (DEFAULT_TABLE table=default.Table_316b63d9aec64a06b194dca118966b47 [id=077d81b3977448b2ae212436d5014339]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.336956 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d5c68c16c3e4d37be7f9a505ab44549. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.337100 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a5415a4246c47ca896751f2fd68af71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.337072 8115 tablet_service.cc:1505] Processing CreateTablet for tablet 39f0a9883b584083b542d078ea7c165b (DEFAULT_TABLE table=default.Table_b2e295519dda4780ba93dede43e9b41c [id=5de7688626854f0c8cabe8fb32b28ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.337252 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 5fa32a1433cd47a9ad39d1c613a18761 (DEFAULT_TABLE table=default.Table_bc8cc7f063644f3eaa9fb0abce91091a [id=bae2c0c5846d4cf69eeac13da536c4cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.337340 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet ba64524142934af08658e434e7fb08ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.337416 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 4393587f022446828e344a9c394a92f9 (DEFAULT_TABLE table=default.Table_0ce46dd2a59848399a1d4fdeac7deef9 [id=e574f40b6ddb4eccae167d16875b0e7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.337565 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 2fc2faa9c4f04fe3b931ce8a2808d738 (DEFAULT_TABLE table=default.Table_ee1375b2eca6432291e041e9aaf7bf3a [id=eaebbf79fab746dfaa81407d3509306c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.337759 8112 tablet_service.cc:1505] Processing CreateTablet for tablet d37c267aaee7412b869753164556b9c0 (DEFAULT_TABLE table=default.Table_4f7a6b08bb2b4cee90e54afd8f1b801a [id=243b57c791df41819156363e8607623d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.338093 8111 tablet_service.cc:1505] Processing CreateTablet for tablet d195958488bf401c8668116a97bf0f58 (DEFAULT_TABLE table=default.Table_62c4ca6858344881a02383c7919d85cb [id=d95696d7d42c4d999f99b584a7232245]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.338131 8127 tablet_service.cc:1505] Processing CreateTablet for tablet f5e7e0614af04cc2a22130906d062a79 (DEFAULT_TABLE table=default.Table_85b63c813dc94719b07f0e12854b164b [id=49bac53017824831928c69211121863b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.338497 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 0ee59aeedd2c4c9ba94afe6578af649a (DEFAULT_TABLE table=default.Table_033167c60f644376b85c64d13503579f [id=7615c3654d9746ca87d81bdacb360935]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.338543 8552 ts_tablet_manager.cc:1916] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.336295 8125 tablet_service.cc:1505] Processing CreateTablet for tablet f1d727c3dbef467da0aec436ffc393be (DEFAULT_TABLE table=default.Table_fcb100729f5b48b990a9c4d2d9b9fcf2 [id=0381bc31864343c19a461fe49735d58a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.337250 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39f0a9883b584083b542d078ea7c165b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.339514 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fc2faa9c4f04fe3b931ce8a2808d738. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.339596 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 463f1c3ac2d74791830431191601282c (DEFAULT_TABLE table=default.Table_831a31c4a8fe414e8c9e8462b95a612e [id=e04ba046a39944a78de902cd6377c58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.346060 7964 catalog_manager.cc:5649] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:57.346446 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet fef84b69b1b746d4b4feeed96670c67b (table default.Table_2f27286185dc41e4b5bc773a4ccdd68a [id=cf715387170e40c992807874e34c66b7]). Expected version 1 got 0
I20251028 02:59:57.346714 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37c267aaee7412b869753164556b9c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.347509 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4393587f022446828e344a9c394a92f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.348860 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa32a1433cd47a9ad39d1c613a18761. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.349071 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d727c3dbef467da0aec436ffc393be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.349617 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d195958488bf401c8668116a97bf0f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.350836 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20410feb13543db9642c1e40bfb9300. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.351436 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5e7e0614af04cc2a22130906d062a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.353346 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee59aeedd2c4c9ba94afe6578af649a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.354830 8546 tablet_bootstrap.cc:492] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.354933 8546 ts_tablet_manager.cc:1403] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.249s user 0.001s sys 0.002s
I20251028 02:59:57.355340 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 463f1c3ac2d74791830431191601282c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.355513 8546 raft_consensus.cc:359] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.355626 8546 raft_consensus.cc:385] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.355667 8546 raft_consensus.cc:740] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.355784 8546 consensus_queue.cc:260] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.356053 8546 ts_tablet_manager.cc:1434] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.356262 8546 tablet_bootstrap.cc:492] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:57.356828 8354 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20251028 02:59:57.357299 8546 tablet_bootstrap.cc:654] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:57.357955 7956 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 4f90e3c3524e4498aa6c4bfc5304d44b (table default.Table_42f89d7fccbf4c738293e9796be235a0 [id=6f3e689a22ae4160ae7537d955c06a90], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4f90e3c3524e4498aa6c4bfc5304d44b consensus state has no leader
I20251028 02:59:57.369393 8545 tablet_bootstrap.cc:492] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:57.369544 8545 ts_tablet_manager.cc:1403] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.217s user 0.002s sys 0.000s
I20251028 02:59:57.370157 8545 raft_consensus.cc:359] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.370270 8545 raft_consensus.cc:385] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.370306 8545 raft_consensus.cc:740] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.370457 8546 tablet_bootstrap.cc:492] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.370440 8545 consensus_queue.cc:260] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.370545 8546 ts_tablet_manager.cc:1403] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.001s
I20251028 02:59:57.370743 8545 ts_tablet_manager.cc:1434] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.371065 8545 tablet_bootstrap.cc:492] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:57.371188 8546 raft_consensus.cc:359] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.371289 8546 raft_consensus.cc:385] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.371381 8546 raft_consensus.cc:740] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.371515 8546 consensus_queue.cc:260] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.371912 8546 ts_tablet_manager.cc:1434] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251028 02:59:57.372110 8546 tablet_bootstrap.cc:492] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.372767 8545 tablet_bootstrap.cc:654] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.373297 8546 tablet_bootstrap.cc:654] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.373785 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 3435803c6167447a8b3afba6c0f3511b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.377890 8552 ts_tablet_manager.cc:1929] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.377971 8552 log.cc:1199] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9e260030c0e6423cba37a2e3af1c6400
I20251028 02:59:57.378228 8552 ts_tablet_manager.cc:1950] T 9e260030c0e6423cba37a2e3af1c6400 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.378815 8552 tablet_replica.cc:333] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.378960 8552 raft_consensus.cc:2243] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.379144 8552 raft_consensus.cc:2272] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:57.379792 9628 consensus_queue.cc:1175] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20251028 02:59:57.381012 9551 tablet.cc:1722] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:57.381340 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 529b865389f94471a6849d17ff206e4c with delete_type TABLET_DATA_DELETED (replaced by tablet 240b7da6f46c4c5ab8087ab1dd6275c5) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.381433 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.381498 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfc5240893546d7a6a5397cd08044dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.383656 8544 tablet_bootstrap.cc:492] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.383742 8544 ts_tablet_manager.cc:1403] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.117s user 0.002s sys 0.001s
I20251028 02:59:57.382643 9646 raft_consensus.cc:493] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.384217 8544 raft_consensus.cc:359] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.384325 8544 raft_consensus.cc:385] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.384356 8544 raft_consensus.cc:740] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.384467 8544 consensus_queue.cc:260] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.384733 8544 ts_tablet_manager.cc:1434] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:57.384927 8544 tablet_bootstrap.cc:492] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.385991 8544 tablet_bootstrap.cc:654] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.384222 9646 raft_consensus.cc:515] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.386878 9646 leader_election.cc:290] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.387665 8550 ts_tablet_manager.cc:1916] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.388790 8552 ts_tablet_manager.cc:1916] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.389950 8550 ts_tablet_manager.cc:1929] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.390012 8550 log.cc:1199] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/35e211a1a8c547baaee1199f7103bfad
I20251028 02:59:57.390264 8550 ts_tablet_manager.cc:1950] T 35e211a1a8c547baaee1199f7103bfad P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.390542 8550 tablet_replica.cc:333] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.390657 8550 raft_consensus.cc:2243] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.390759 8550 raft_consensus.cc:2272] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.391386 8550 ts_tablet_manager.cc:1916] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.393615 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6dfc5240893546d7a6a5397cd08044dd" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.393806 8550 ts_tablet_manager.cc:1929] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.393882 8550 log.cc:1199] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/26a344e2e03e4eeaa688275e9ecbf63f
I20251028 02:59:57.394121 8550 ts_tablet_manager.cc:1950] T 26a344e2e03e4eeaa688275e9ecbf63f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.394377 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6dfc5240893546d7a6a5397cd08044dd" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.394517 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 698cca4847e94a79933c644f6b1c1c8d (DEFAULT_TABLE table=default.Table_272a84b9ac3e46c882522d75055385d3 [id=507a066c80f643a9b637d3a8f01e8228]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.394760 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 698cca4847e94a79933c644f6b1c1c8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.395068 8550 tablet_replica.cc:333] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.395196 8550 raft_consensus.cc:2243] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.395418 9646 raft_consensus.cc:493] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.395488 9646 raft_consensus.cc:515] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.395869 9646 leader_election.cc:290] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.395974 8546 tablet_bootstrap.cc:492] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.395999 8550 raft_consensus.cc:2272] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.396075 8546 ts_tablet_manager.cc:1403] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.002s
I20251028 02:59:57.396557 8550 ts_tablet_manager.cc:1916] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.396589 8546 raft_consensus.cc:359] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.396682 8546 raft_consensus.cc:385] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.396745 8546 raft_consensus.cc:740] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.396885 8546 consensus_queue.cc:260] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.397404 8544 tablet_bootstrap.cc:492] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.397490 8544 ts_tablet_manager.cc:1403] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20251028 02:59:57.397979 8544 raft_consensus.cc:359] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.397960 9595 consensus_queue.cc:1048] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.398063 8544 raft_consensus.cc:385] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.398093 8544 raft_consensus.cc:740] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.398186 8544 consensus_queue.cc:260] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.398460 8544 ts_tablet_manager.cc:1434] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:57.399061 9529 tablet.cc:1722] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.399682 8550 ts_tablet_manager.cc:1929] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.399825 8550 log.cc:1199] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/551aea93ba064bc89ae00b002eb7ccfc
W20251028 02:59:57.399998 8083 leader_election.cc:343] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.400395 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6ac7b5fff0848cebf710ba3ed50def4" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.400528 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6ac7b5fff0848cebf710ba3ed50def4" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:57.404525 8083 leader_election.cc:343] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.404673 8083 leader_election.cc:304] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.404830 8546 ts_tablet_manager.cc:1434] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
W20251028 02:59:57.404940 8083 leader_election.cc:343] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:57.405055 8083 leader_election.cc:343] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.405100 8083 leader_election.cc:304] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.405225 9650 raft_consensus.cc:2749] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.405326 9650 raft_consensus.cc:2749] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.406335 8546 tablet_bootstrap.cc:492] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.407398 8546 tablet_bootstrap.cc:654] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.408138 8552 ts_tablet_manager.cc:1929] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.408212 8552 log.cc:1199] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/72350c60d73b41829d99ac8d01aa04bc
I20251028 02:59:57.408486 8552 ts_tablet_manager.cc:1950] T 72350c60d73b41829d99ac8d01aa04bc P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.415360 9595 consensus_queue.cc:1048] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.418342 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]). Expected version 1 got 0
W20251028 02:59:57.423113 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 6dd563e13e624d4bb88950b46d480876 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6dd563e13e624d4bb88950b46d480876 already in progress: creating tablet
W20251028 02:59:57.423491 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e19a2bdc7024b0c9fe84f8983f9d292 already in progress: creating tablet
I20251028 02:59:57.423523 8546 tablet_bootstrap.cc:492] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.423620 8546 ts_tablet_manager.cc:1403] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
W20251028 02:59:57.423640 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 3435803c6167447a8b3afba6c0f3511b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3435803c6167447a8b3afba6c0f3511b already in progress: creating tablet
W20251028 02:59:57.423738 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 597d792ed13941a2a895b0dc305614f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 597d792ed13941a2a895b0dc305614f1 already in progress: creating tablet
I20251028 02:59:57.423952 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9e260030c0e6423cba37a2e3af1c6400 (table default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]) successfully deleted
I20251028 02:59:57.424060 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 72350c60d73b41829d99ac8d01aa04bc (table default.Table_b2e295519dda4780ba93dede43e9b41c [id=5de7688626854f0c8cabe8fb32b28ede]) successfully deleted
W20251028 02:59:57.424165 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ba64524142934af08658e434e7fb08ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba64524142934af08658e434e7fb08ed already in progress: creating tablet
I20251028 02:59:57.424129 8546 raft_consensus.cc:359] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.424233 8546 raft_consensus.cc:385] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.424271 8546 raft_consensus.cc:740] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.424376 8546 consensus_queue.cc:260] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.424626 8546 ts_tablet_manager.cc:1434] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.424829 8546 tablet_bootstrap.cc:492] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:57.424885 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 3435803c6167447a8b3afba6c0f3511b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3435803c6167447a8b3afba6c0f3511b already in progress: creating tablet
I20251028 02:59:57.425164 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_07cff4b56d204f6483cbe27f181cc80a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.425367 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_81729091394d4eeda0b760c59680ae19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.425474 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3249352b73e544a1afa41660422fe744"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.425555 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_07cff4b56d204f6483cbe27f181cc80a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:57.425704 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_81729091394d4eeda0b760c59680ae19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:57.425751 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3249352b73e544a1afa41660422fe744 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.425892 8546 tablet_bootstrap.cc:654] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.426324 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_af589bb054d049a8beaed83cc51499f5" } new_table_name: "default.Table_e2683e2b014947bd87e048f63aa666f1" modify_external_catalogs: true
I20251028 02:59:57.426532 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_aeee4c7fa7ae41beb0ef15ad542213e8" } new_table_name: "default.Table_9f07958f85cf4923a67bf8ac1ad1c722" modify_external_catalogs: true
I20251028 02:59:57.426680 9595 consensus_queue.cc:1048] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.426862 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 698cca4847e94a79933c644f6b1c1c8d (DEFAULT_TABLE table=default.Table_272a84b9ac3e46c882522d75055385d3 [id=507a066c80f643a9b637d3a8f01e8228]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.426662 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet cbecb6472dc54989beca91ca7bcb0b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.427129 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 698cca4847e94a79933c644f6b1c1c8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.428880 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 35e211a1a8c547baaee1199f7103bfad (table default.Table_26410f48ded64d1285940aeff4f1fc07 [id=3191deea26744b009e341282c1babcbf]) successfully deleted
I20251028 02:59:57.428998 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 26a344e2e03e4eeaa688275e9ecbf63f (table default.Table_152433eb6e11487198b81047f13cc1f1 [id=8642a2095b8049de90e6df3a8570d730]) successfully deleted
I20251028 02:59:57.430773 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a290b8114844aa89b33c75fffc8e1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.431143 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 698cca4847e94a79933c644f6b1c1c8d (DEFAULT_TABLE table=default.Table_272a84b9ac3e46c882522d75055385d3 [id=507a066c80f643a9b637d3a8f01e8228]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.432533 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]). Expected version 2 got 0
I20251028 02:59:57.434218 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 698cca4847e94a79933c644f6b1c1c8d. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:57.436779 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ff112c66f1a4cf9b232e6bb665755f9 consensus state has no leader
I20251028 02:59:57.437192 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet f6ac7b5fff0848cebf710ba3ed50def4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f90e3c3524e4498aa6c4bfc5304d44b) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:57.438650 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 02695b18936d4f928cd55f9b76b90b2e consensus state has no leader
I20251028 02:59:57.443646 9596 raft_consensus.cc:493] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.443648 9595 raft_consensus.cc:493] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.443815 9596 raft_consensus.cc:515] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.443815 9595 raft_consensus.cc:515] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.444232 9596 leader_election.cc:290] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.444232 9595 leader_election.cc:290] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.444831 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fcab55fd49da41f7a95d46d76a8dac8b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.444929 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5428db004de74f7492c9b8999e53020e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:57.445052 8147 raft_consensus.cc:2468] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:57.445132 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5428db004de74f7492c9b8999e53020e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.445350 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fcab55fd49da41f7a95d46d76a8dac8b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:57.445448 8143 raft_consensus.cc:2468] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
W20251028 02:59:57.445879 8220 leader_election.cc:343] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:57.446034 8220 leader_election.cc:343] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.447701 8218 leader_election.cc:304] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.447947 9595 raft_consensus.cc:2804] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:57.448001 9595 raft_consensus.cc:493] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:57.448056 9595 raft_consensus.cc:3060] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.449172 9595 raft_consensus.cc:515] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.449530 9595 leader_election.cc:290] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.449859 8218 leader_election.cc:304] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.449910 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5428db004de74f7492c9b8999e53020e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
W20251028 02:59:57.450268 8220 leader_election.cc:343] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.450295 9595 raft_consensus.cc:2804] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:57.450311 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5428db004de74f7492c9b8999e53020e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:57.450356 9595 raft_consensus.cc:493] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:57.450397 9595 raft_consensus.cc:3060] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.450428 8143 raft_consensus.cc:3060] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.451407 8552 tablet_replica.cc:333] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.451431 9595 raft_consensus.cc:515] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.451495 8143 raft_consensus.cc:2468] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:57.451525 8552 raft_consensus.cc:2243] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.451611 8552 raft_consensus.cc:2272] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.451794 9595 leader_election.cc:290] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.452080 8218 leader_election.cc:304] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.452178 8552 ts_tablet_manager.cc:1916] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.452200 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fcab55fd49da41f7a95d46d76a8dac8b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:57.452489 9596 raft_consensus.cc:2804] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:57.452575 9596 raft_consensus.cc:697] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
W20251028 02:59:57.452572 8220 leader_election.cc:343] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.452737 9596 consensus_queue.cc:237] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.452824 9595 raft_consensus.cc:493] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.452880 9595 raft_consensus.cc:515] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.452881 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fcab55fd49da41f7a95d46d76a8dac8b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:57.452978 8143 raft_consensus.cc:3060] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.453157 9595 leader_election.cc:290] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.453648 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a290b8114844aa89b33c75fffc8e1e7" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:57.453759 8147 raft_consensus.cc:2468] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:57.453928 8143 raft_consensus.cc:2468] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:57.454075 8218 leader_election.cc:304] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:57.454316 8218 leader_election.cc:304] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.454979 9594 raft_consensus.cc:2804] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:57.455047 9594 raft_consensus.cc:493] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:57.455086 9594 raft_consensus.cc:3060] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.455806 9595 raft_consensus.cc:2804] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:57.456391 9595 raft_consensus.cc:697] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:57.456517 9595 consensus_queue.cc:237] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.457726 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a290b8114844aa89b33c75fffc8e1e7" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.458809 9594 raft_consensus.cc:515] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.459141 9594 leader_election.cc:290] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:57.459682 8220 leader_election.cc:343] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.459990 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a290b8114844aa89b33c75fffc8e1e7" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:57.460108 8145 raft_consensus.cc:3060] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.460922 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a290b8114844aa89b33c75fffc8e1e7" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:57.461057 8145 raft_consensus.cc:2468] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
W20251028 02:59:57.461249 8220 leader_election.cc:343] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.461426 8218 leader_election.cc:304] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.461637 9594 raft_consensus.cc:2804] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:57.461714 9594 raft_consensus.cc:697] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:57.461826 9594 consensus_queue.cc:237] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.462877 9639 raft_consensus.cc:493] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.462970 9639 raft_consensus.cc:515] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.463349 9639 leader_election.cc:290] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.463605 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13de247dfb18478daac9ca5601220faa" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:57.463721 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13de247dfb18478daac9ca5601220faa" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.463768 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2681fbe951fb4a88bbf4c192aac594d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.463871 8271 raft_consensus.cc:2393] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7e4cf694f71438298dd32a346abcf75 in current term 1: Already voted for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in this term.
I20251028 02:59:57.464201 8354 leader_election.cc:304] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:57.464260 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2681fbe951fb4a88bbf4c192aac594d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.464406 9639 raft_consensus.cc:3060] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.465425 9639 raft_consensus.cc:2749] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.454991 8552 ts_tablet_manager.cc:1929] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.465819 8552 log.cc:1199] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/cbecb6472dc54989beca91ca7bcb0b57
I20251028 02:59:57.463781 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b6f5cef4fc824b09ab85a48c5bd021a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.466207 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b6f5cef4fc824b09ab85a48c5bd021a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.466254 8552 ts_tablet_manager.cc:1950] T cbecb6472dc54989beca91ca7bcb0b57 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.467024 8552 tablet_replica.cc:333] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.474946 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]). Expected version 1 got 0
I20251028 02:59:57.469872 8546 tablet_bootstrap.cc:492] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.475278 8546 ts_tablet_manager.cc:1403] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.001s
I20251028 02:59:57.475893 8546 raft_consensus.cc:359] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.476011 8546 raft_consensus.cc:385] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.476050 8546 raft_consensus.cc:740] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.476176 8546 consensus_queue.cc:260] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.476483 8546 ts_tablet_manager.cc:1434] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:57.476670 8546 tablet_bootstrap.cc:492] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.477885 8546 tablet_bootstrap.cc:654] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:57.474970 8220 consensus_peers.cc:597] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:57.480063 8552 raft_consensus.cc:2243] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.480327 8552 pending_rounds.cc:70] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:57.480444 8552 pending_rounds.cc:77] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597148004839424 op_type: NO_OP noop_request { }
I20251028 02:59:57.480561 8552 raft_consensus.cc:2889] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:57.480679 8552 raft_consensus.cc:2272] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.480897 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7c8e2218f01e4adbbff3b276fe5345d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.481364 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c8e2218f01e4adbbff3b276fe5345d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.481420 8552 ts_tablet_manager.cc:1916] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.483934 8552 ts_tablet_manager.cc:1929] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.484021 8552 log.cc:1199] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9a290b8114844aa89b33c75fffc8e1e7
I20251028 02:59:57.484285 8552 ts_tablet_manager.cc:1950] T 9a290b8114844aa89b33c75fffc8e1e7 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.485494 8546 tablet_bootstrap.cc:492] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.485585 8546 ts_tablet_manager.cc:1403] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20251028 02:59:57.486114 8546 raft_consensus.cc:359] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.486214 8546 raft_consensus.cc:385] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.486246 8546 raft_consensus.cc:740] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.486361 8546 consensus_queue.cc:260] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.486644 8546 ts_tablet_manager.cc:1434] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.486841 8546 tablet_bootstrap.cc:492] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.488145 8546 tablet_bootstrap.cc:654] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.489471 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_19f5792cad7f4b2fa7f0446f6c3c7bb0" } modify_external_catalogs: true
I20251028 02:59:57.489681 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_19f5792cad7f4b2fa7f0446f6c3c7bb0" } modify_external_catalogs: true
I20251028 02:59:57.498308 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet b6d868c81a7e402ba9305bb2aa90625b
I20251028 02:59:57.499181 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet b6d868c81a7e402ba9305bb2aa90625b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.500723 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ef415299e6ef4aa583629acaba69daf0" } modify_external_catalogs: true
I20251028 02:59:57.500845 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ef415299e6ef4aa583629acaba69daf0" } modify_external_catalogs: true
I20251028 02:59:57.501395 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_594bc3babcbf48e4b5d9490f18936563" } modify_external_catalogs: true
I20251028 02:59:57.501439 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d0d760998d424df696eb71d3f8223366" } new_table_name: "default.Table_f1e09c9160e64a2bad978b4ebabb2c8f" modify_external_catalogs: true
I20251028 02:59:57.501508 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_594bc3babcbf48e4b5d9490f18936563" } modify_external_catalogs: true
I20251028 02:59:57.510375 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_206be08f6668466db643e3bf2fff50ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.510987 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_206be08f6668466db643e3bf2fff50ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.519364 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b2e295519dda4780ba93dede43e9b41c" } modify_external_catalogs: true
I20251028 02:59:57.519543 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b2e295519dda4780ba93dede43e9b41c" } modify_external_catalogs: true
I20251028 02:59:57.520107 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_87426eb930ad44e5902eb3aee3da0d71" } new_table_name: "default.Table_01d864857be34675882e1be7ba5a9a7c" modify_external_catalogs: true
I20251028 02:59:57.520203 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6eabdbe1000e4fb6b82c9425cc2e0edd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.520568 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6eabdbe1000e4fb6b82c9425cc2e0edd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.520933 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_78fb1bab55774710bbc8014a9085fb0d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.521217 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78fb1bab55774710bbc8014a9085fb0d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.521799 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet f1d727c3dbef467da0aec436ffc393be from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.523031 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 2893b3a6e8854187b837a7e6fef58c4e from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.523563 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet b6d868c81a7e402ba9305bb2aa90625b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.523967 8552 tablet_replica.cc:333] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.524144 8552 raft_consensus.cc:2243] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.524360 8552 raft_consensus.cc:2272] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.526219 9526 tablet.cc:1722] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.526286 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet b6d868c81a7e402ba9305bb2aa90625b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.526744 8552 ts_tablet_manager.cc:1916] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.527119 8551 tablet_replica.cc:333] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.527261 8551 raft_consensus.cc:2243] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.527392 8551 raft_consensus.cc:2272] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.528041 8551 ts_tablet_manager.cc:1916] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:57.530170 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8e202e53053248a7a9cee6a217999489 consensus state has no leader
I20251028 02:59:57.530360 8551 ts_tablet_manager.cc:1929] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:57.530367 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 2fec0adb79d04469b90c1ea2a5e9ba5b from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.530427 8551 log.cc:1199] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/b6d868c81a7e402ba9305bb2aa90625b
I20251028 02:59:57.530689 8551 ts_tablet_manager.cc:1950] T b6d868c81a7e402ba9305bb2aa90625b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.530759 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 07d3e6c640434652a010942402c54716 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.531651 9529 tablet.cc:1722] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.534214 8552 ts_tablet_manager.cc:1929] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:57.534298 8552 log.cc:1199] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/b6d868c81a7e402ba9305bb2aa90625b
I20251028 02:59:57.535708 9551 tablet.cc:1722] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.537197 8552 ts_tablet_manager.cc:1950] T b6d868c81a7e402ba9305bb2aa90625b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.538955 8553 raft_consensus.cc:493] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.539047 8553 raft_consensus.cc:515] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.539580 8553 leader_election.cc:290] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.540230 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9583046f549d411a9e341560f428381c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:57.542382 8220 leader_election.cc:343] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.543092 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 07d3e6c640434652a010942402c54716
I20251028 02:59:57.543370 7966 catalog_manager.cc:6719] ReplaceTablet: tablet 07d3e6c640434652a010942402c54716 deleted and replaced by tablet 7e0e038d33bf45f693da251cd560ae2a
I20251028 02:59:57.543488 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.544108 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9583046f549d411a9e341560f428381c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:57.544245 8145 raft_consensus.cc:2468] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:57.546667 7963 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9] alter complete (version 1)
I20251028 02:59:57.546976 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f623537faf00450f8b0997cb0e2ab81b" } new_table_name: "default.Table_a01151cafd944efcbc12e5f654a7a05a" modify_external_catalogs: true
I20251028 02:59:57.549811 8218 leader_election.cc:304] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.550040 8553 raft_consensus.cc:2804] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:57.550088 8553 raft_consensus.cc:493] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:57.550123 8553 raft_consensus.cc:3060] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.551182 8553 raft_consensus.cc:515] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.551584 8553 leader_election.cc:290] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.552136 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9583046f549d411a9e341560f428381c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:57.552542 9551 tablet.cc:1722] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251028 02:59:57.552619 8220 leader_election.cc:343] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.554431 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9583046f549d411a9e341560f428381c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:57.554551 8144 raft_consensus.cc:3060] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:57.555542 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f422030a6a4042009efc7fcfb21a6bca consensus state has no leader
I20251028 02:59:57.555552 8144 raft_consensus.cc:2468] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:57.555756 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8c2ee567a0b74836997f0d500266c256" } modify_external_catalogs: true
I20251028 02:59:57.555850 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8c2ee567a0b74836997f0d500266c256" } modify_external_catalogs: true
I20251028 02:59:57.555856 8218 leader_election.cc:304] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.556034 8553 raft_consensus.cc:2804] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:57.556109 8553 raft_consensus.cc:697] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:57.556228 8553 consensus_queue.cc:237] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.557085 9529 tablet.cc:1722] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.558568 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 07d3e6c640434652a010942402c54716 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e0e038d33bf45f693da251cd560ae2a) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.561966 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_feec1045abaf4ffb9ef0e698d450a0c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.562673 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_feec1045abaf4ffb9ef0e698d450a0c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.570860 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:57.571053 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]). Expected version 1 got 0
I20251028 02:59:57.574295 7964 catalog_manager.cc:5649] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:57.574518 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]). Expected version 2 got 0
I20251028 02:59:57.577009 9646 raft_consensus.cc:493] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.577122 9646 raft_consensus.cc:515] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.577471 9646 leader_election.cc:290] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.578701 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35d5e3fa27ab46f9823b9cf4f7b3eb2a" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.579023 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35d5e3fa27ab46f9823b9cf4f7b3eb2a" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:57.579583 8083 leader_election.cc:343] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:57.579917 8083 leader_election.cc:343] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.579988 8083 leader_election.cc:304] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.580251 9646 raft_consensus.cc:2749] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.584960 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a
I20251028 02:59:57.585110 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]). Expected version 1 got 0
I20251028 02:59:57.585377 7966 catalog_manager.cc:6719] ReplaceTablet: tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a deleted and replaced by tablet 4eaa472029a94052abb49241b7622d0f
I20251028 02:59:57.586467 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a with delete_type TABLET_DATA_DELETED (replaced by tablet 4eaa472029a94052abb49241b7622d0f) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.592697 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2fec0adb79d04469b90c1ea2a5e9ba5b
I20251028 02:59:57.592983 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 2fec0adb79d04469b90c1ea2a5e9ba5b deleted and replaced by tablet f898e9072d7d4697aee6a981cf6188b4
I20251028 02:59:57.594381 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 7b552f63895449aea79cbd8867cc063c
I20251028 02:59:57.595311 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 39f0a9883b584083b542d078ea7c165b
I20251028 02:59:57.595356 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f1d727c3dbef467da0aec436ffc393be
I20251028 02:59:57.595489 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c0f428e5c6fe43fc8f78761d868ddd09
I20251028 02:59:57.595645 7968 catalog_manager.cc:6719] ReplaceTablet: tablet f1d727c3dbef467da0aec436ffc393be deleted and replaced by tablet a0943be517fd49e199fd060782a22c31
I20251028 02:59:57.596272 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8192667f14a44ec296d28efa6057a44d
I20251028 02:59:57.597265 7963 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet a808098c70b54ad3a0230352af0b743c (table default.Table_9e950f6a07524cb0b5a0d3d967668ad6 [id=5dd4cb94fa384facbdd1aef677a4b498]). Expected version 1 got 0
I20251028 02:59:57.597360 7963 catalog_manager.cc:5649] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:57.597589 7963 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]). Expected version 1 got 0
I20251028 02:59:57.598402 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b with delete_type TABLET_DATA_DELETED (replaced by tablet f898e9072d7d4697aee6a981cf6188b4) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.598593 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b552f63895449aea79cbd8867cc063c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.599002 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet f1d727c3dbef467da0aec436ffc393be with delete_type TABLET_DATA_DELETED (replaced by tablet a0943be517fd49e199fd060782a22c31) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.599085 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 39f0a9883b584083b542d078ea7c165b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.599237 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet c0f428e5c6fe43fc8f78761d868ddd09 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.403931 8544 tablet_bootstrap.cc:492] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.599272 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 8192667f14a44ec296d28efa6057a44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.600396 8544 tablet_bootstrap.cc:654] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.601392 8544 tablet_bootstrap.cc:492] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.601475 8544 ts_tablet_manager.cc:1403] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.198s user 0.001s sys 0.000s
I20251028 02:59:57.601873 8544 raft_consensus.cc:359] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.601979 8544 raft_consensus.cc:385] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.602005 8544 raft_consensus.cc:740] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.602113 8544 consensus_queue.cc:260] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.602368 8544 ts_tablet_manager.cc:1434] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.602561 8544 tablet_bootstrap.cc:492] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.603494 8544 tablet_bootstrap.cc:654] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.605937 8544 tablet_bootstrap.cc:492] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.606024 8544 ts_tablet_manager.cc:1403] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20251028 02:59:57.606495 8544 raft_consensus.cc:359] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.606598 8544 raft_consensus.cc:385] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.606628 8544 raft_consensus.cc:740] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.606736 8544 consensus_queue.cc:260] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.607039 8544 ts_tablet_manager.cc:1434] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:57.607218 8544 tablet_bootstrap.cc:492] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.607465 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_f1e09c9160e64a2bad978b4ebabb2c8f [id=f1872634680f477e9f44007a30ef02bd] alter complete (version 1)
I20251028 02:59:57.400161 8550 ts_tablet_manager.cc:1950] T 551aea93ba064bc89ae00b002eb7ccfc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.608274 8544 tablet_bootstrap.cc:654] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.612038 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 551aea93ba064bc89ae00b002eb7ccfc (table default.Table_806d85836a4c44a986095f5108a93d7f [id=c098c2727dd147e69471353bd9a741fc]) successfully deleted
I20251028 02:59:57.613045 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet cbecb6472dc54989beca91ca7bcb0b57 (table default.Table_3292bf8a27264e8491009af851138333 [id=d317a63dea224c45b4dfd1a205ae4c68]) successfully deleted
I20251028 02:59:57.613173 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9a290b8114844aa89b33c75fffc8e1e7 (table default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]) successfully deleted
I20251028 02:59:57.613157 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 72c8f9833434439da92711fe04e838af (DEFAULT_TABLE table=default.Table_81729091394d4eeda0b760c59680ae19 [id=96dd6ba48a454763ae11133ee8fdb0a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.613265 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet b6d868c81a7e402ba9305bb2aa90625b (table default.Table_19f5792cad7f4b2fa7f0446f6c3c7bb0 [id=da79674aa8cd468e84114457ee46766e]) successfully deleted
I20251028 02:59:57.613451 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c8f9833434439da92711fe04e838af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.613476 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet b6d868c81a7e402ba9305bb2aa90625b (table default.Table_19f5792cad7f4b2fa7f0446f6c3c7bb0 [id=da79674aa8cd468e84114457ee46766e]) successfully deleted
I20251028 02:59:57.614347 8550 tablet_replica.cc:333] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.614502 8550 raft_consensus.cc:2243] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.614616 8550 raft_consensus.cc:2272] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.615653 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 07d3e6c640434652a010942402c54716 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e0e038d33bf45f693da251cd560ae2a) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.615715 8544 tablet_bootstrap.cc:492] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.615792 8544 ts_tablet_manager.cc:1403] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 02:59:57.615940 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 07d3e6c640434652a010942402c54716 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e0e038d33bf45f693da251cd560ae2a) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.616338 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a with delete_type TABLET_DATA_DELETED (replaced by tablet 4eaa472029a94052abb49241b7622d0f) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.616302 8544 raft_consensus.cc:359] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.616405 8544 raft_consensus.cc:385] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.616441 8544 raft_consensus.cc:740] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.616494 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet bd50e7d4e28541de9799833c53fe7c02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.616561 8544 consensus_queue.cc:260] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.616376 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 874b6660d86b48ebad03ecc54588da28 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d3e422c7ae64b25960714ecc926046a) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.616852 8544 ts_tablet_manager.cc:1434] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.616916 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 72c8f9833434439da92711fe04e838af (DEFAULT_TABLE table=default.Table_81729091394d4eeda0b760c59680ae19 [id=96dd6ba48a454763ae11133ee8fdb0a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.617044 8544 tablet_bootstrap.cc:492] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.617192 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c8f9833434439da92711fe04e838af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.618057 8544 tablet_bootstrap.cc:654] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.618721 7965 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet b46554b0df894beb8141126aa7ee7b8e (table default.Table_176b16f7b1fd4a30991b82efd4ddd497 [id=9e5ad11a9ea34a3a8c629f48142b68f0]). Expected version 1 got 0
I20251028 02:59:57.621771 8550 ts_tablet_manager.cc:1916] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.624181 8550 ts_tablet_manager.cc:1929] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.624405 8550 log.cc:1199] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/cbecb6472dc54989beca91ca7bcb0b57
I20251028 02:59:57.624706 8550 ts_tablet_manager.cc:1950] T cbecb6472dc54989beca91ca7bcb0b57 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.625407 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet cbecb6472dc54989beca91ca7bcb0b57 (table default.Table_3292bf8a27264e8491009af851138333 [id=d317a63dea224c45b4dfd1a205ae4c68]) successfully deleted
I20251028 02:59:57.625674 8550 tablet_replica.cc:333] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.625837 8550 raft_consensus.cc:2243] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.625970 8550 raft_consensus.cc:2272] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.626191 7965 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet b46554b0df894beb8141126aa7ee7b8e (table default.Table_176b16f7b1fd4a30991b82efd4ddd497 [id=9e5ad11a9ea34a3a8c629f48142b68f0]). Expected version 1 got 0
I20251028 02:59:57.626317 8550 ts_tablet_manager.cc:1916] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.628266 8550 ts_tablet_manager.cc:1929] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.628343 8550 log.cc:1199] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9a290b8114844aa89b33c75fffc8e1e7
I20251028 02:59:57.628530 8550 ts_tablet_manager.cc:1950] T 9a290b8114844aa89b33c75fffc8e1e7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.629170 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9a290b8114844aa89b33c75fffc8e1e7 (table default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]) successfully deleted
I20251028 02:59:57.629649 8544 tablet_bootstrap.cc:492] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.629765 8544 ts_tablet_manager.cc:1403] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20251028 02:59:57.630244 8544 raft_consensus.cc:359] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.630357 8544 raft_consensus.cc:385] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.630393 8544 raft_consensus.cc:740] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.630501 8544 consensus_queue.cc:260] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.630764 8544 ts_tablet_manager.cc:1434] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.635589 8544 tablet_bootstrap.cc:492] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.636708 8544 tablet_bootstrap.cc:654] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:57.637667 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 28b81f0000c4407096feff773c29579d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28b81f0000c4407096feff773c29579d already in progress: creating tablet
W20251028 02:59:57.637904 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 2893b3a6e8854187b837a7e6fef58c4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2893b3a6e8854187b837a7e6fef58c4e already in progress: creating tablet
W20251028 02:59:57.638105 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 8483cbc3099644d2b6f7ff52869e21ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8483cbc3099644d2b6f7ff52869e21ea already in progress: creating tablet
I20251028 02:59:57.638144 8550 tablet_replica.cc:333] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
W20251028 02:59:57.638266 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e19a2bdc7024b0c9fe84f8983f9d292 already in progress: creating tablet
I20251028 02:59:57.638343 8550 raft_consensus.cc:2243] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:57.638401 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6dd563e13e624d4bb88950b46d480876 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6dd563e13e624d4bb88950b46d480876 already in progress: creating tablet
I20251028 02:59:57.638521 8550 raft_consensus.cc:2272] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.639158 8550 ts_tablet_manager.cc:1916] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.640098 8545 tablet_bootstrap.cc:492] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:57.640210 8545 ts_tablet_manager.cc:1403] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.269s user 0.002s sys 0.000s
I20251028 02:59:57.640704 8545 raft_consensus.cc:359] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.640826 8545 raft_consensus.cc:385] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.640862 8545 raft_consensus.cc:740] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.640993 8545 consensus_queue.cc:260] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.641211 8545 ts_tablet_manager.cc:1434] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.641424 8545 tablet_bootstrap.cc:492] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:57.641546 8550 ts_tablet_manager.cc:1929] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.641613 8550 log.cc:1199] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/72350c60d73b41829d99ac8d01aa04bc
I20251028 02:59:57.641888 8550 ts_tablet_manager.cc:1950] T 72350c60d73b41829d99ac8d01aa04bc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.642204 8545 tablet_bootstrap.cc:654] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:57.642804 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 3435803c6167447a8b3afba6c0f3511b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3435803c6167447a8b3afba6c0f3511b already in progress: creating tablet
I20251028 02:59:57.642922 8550 tablet_replica.cc:333] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
W20251028 02:59:57.642982 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 597d792ed13941a2a895b0dc305614f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 597d792ed13941a2a895b0dc305614f1 already in progress: creating tablet
I20251028 02:59:57.643078 8550 raft_consensus.cc:2243] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.643222 8550 raft_consensus.cc:2272] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.643966 8550 ts_tablet_manager.cc:1916] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.645344 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a with delete_type TABLET_DATA_DELETED (replaced by tablet 4eaa472029a94052abb49241b7622d0f) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.645813 9526 tablet.cc:1722] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.646922 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b with delete_type TABLET_DATA_DELETED (replaced by tablet f898e9072d7d4697aee6a981cf6188b4) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.647183 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b with delete_type TABLET_DATA_DELETED (replaced by tablet f898e9072d7d4697aee6a981cf6188b4) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.647634 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b552f63895449aea79cbd8867cc063c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.647727 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b552f63895449aea79cbd8867cc063c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.648343 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet c0f428e5c6fe43fc8f78761d868ddd09 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.648667 8544 tablet_bootstrap.cc:492] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.648773 8544 ts_tablet_manager.cc:1403] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.001s
I20251028 02:59:57.648828 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 39f0a9883b584083b542d078ea7c165b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.649330 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 8192667f14a44ec296d28efa6057a44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.649348 8544 raft_consensus.cc:359] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.649477 8544 raft_consensus.cc:385] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.649538 8544 raft_consensus.cc:740] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
W20251028 02:59:57.649508 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 07d3e6c640434652a010942402c54716 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07d3e6c640434652a010942402c54716 already in progress: creating tablet
W20251028 02:59:57.649667 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a already in progress: creating tablet
I20251028 02:59:57.649659 8544 consensus_queue.cc:260] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.649684 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 72c8f9833434439da92711fe04e838af (DEFAULT_TABLE table=default.Table_81729091394d4eeda0b760c59680ae19 [id=96dd6ba48a454763ae11133ee8fdb0a9]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:57.649817 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2fec0adb79d04469b90c1ea2a5e9ba5b already in progress: creating tablet
W20251028 02:59:57.649930 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 7b552f63895449aea79cbd8867cc063c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b552f63895449aea79cbd8867cc063c already in progress: creating tablet
I20251028 02:59:57.649971 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72c8f9833434439da92711fe04e838af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.650009 8544 ts_tablet_manager.cc:1434] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 02:59:57.650049 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet c0f428e5c6fe43fc8f78761d868ddd09 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c0f428e5c6fe43fc8f78761d868ddd09 already in progress: creating tablet
W20251028 02:59:57.650161 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 39f0a9883b584083b542d078ea7c165b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 39f0a9883b584083b542d078ea7c165b already in progress: creating tablet
I20251028 02:59:57.650214 8544 tablet_bootstrap.cc:492] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 02:59:57.650254 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 8192667f14a44ec296d28efa6057a44d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8192667f14a44ec296d28efa6057a44d already in progress: creating tablet
W20251028 02:59:57.650377 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 07d3e6c640434652a010942402c54716 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07d3e6c640434652a010942402c54716 already in progress: creating tablet
W20251028 02:59:57.650503 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a already in progress: creating tablet
W20251028 02:59:57.650583 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2fec0adb79d04469b90c1ea2a5e9ba5b already in progress: creating tablet
I20251028 02:59:57.651211 8544 tablet_bootstrap.cc:654] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.651599 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet 9583046f549d411a9e341560f428381c with delete_type TABLET_DATA_DELETED (replaced by tablet ba64524142934af08658e434e7fb08ed) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.651808 8552 tablet_replica.cc:333] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.651944 8552 raft_consensus.cc:2243] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.652125 8552 pending_rounds.cc:70] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:57.652156 8552 pending_rounds.cc:77] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597148391661568 op_type: NO_OP noop_request { }
I20251028 02:59:57.652209 8552 raft_consensus.cc:2889] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:57.652258 8552 raft_consensus.cc:2272] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.653128 8552 ts_tablet_manager.cc:1916] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.653349 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet f1d727c3dbef467da0aec436ffc393be with delete_type TABLET_DATA_DELETED (replaced by tablet a0943be517fd49e199fd060782a22c31) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.653813 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet c0f428e5c6fe43fc8f78761d868ddd09 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.653959 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 8192667f14a44ec296d28efa6057a44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.653981 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet 39f0a9883b584083b542d078ea7c165b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.654064 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 04d271c95c2847ca8b1fd2d55741117c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.654383 8551 tablet_replica.cc:333] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.654506 8551 raft_consensus.cc:2243] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.654615 8551 raft_consensus.cc:2272] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.655259 8550 ts_tablet_manager.cc:1929] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.655351 8550 log.cc:1199] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9e260030c0e6423cba37a2e3af1c6400
I20251028 02:59:57.655601 8550 ts_tablet_manager.cc:1950] T 9e260030c0e6423cba37a2e3af1c6400 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.656533 8546 tablet_bootstrap.cc:492] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.656620 8546 ts_tablet_manager.cc:1403] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.170s user 0.000s sys 0.002s
I20251028 02:59:57.656636 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet f1d727c3dbef467da0aec436ffc393be with delete_type TABLET_DATA_DELETED (replaced by tablet a0943be517fd49e199fd060782a22c31) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.657130 8546 raft_consensus.cc:359] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.657238 8546 raft_consensus.cc:385] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.657271 8546 raft_consensus.cc:740] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.657409 8551 ts_tablet_manager.cc:1916] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.657384 8546 consensus_queue.cc:260] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.657663 8546 ts_tablet_manager.cc:1434] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:57.657928 8546 tablet_bootstrap.cc:492] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.658663 8552 ts_tablet_manager.cc:1929] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.658721 8552 log.cc:1199] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9583046f549d411a9e341560f428381c
I20251028 02:59:57.658984 8546 tablet_bootstrap.cc:654] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.659003 8552 ts_tablet_manager.cc:1950] T 9583046f549d411a9e341560f428381c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.659713 8551 ts_tablet_manager.cc:1929] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.659772 8551 log.cc:1199] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/04d271c95c2847ca8b1fd2d55741117c
I20251028 02:59:57.659984 8551 ts_tablet_manager.cc:1950] T 04d271c95c2847ca8b1fd2d55741117c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.667336 8550 tablet_replica.cc:333] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.667526 8550 raft_consensus.cc:2243] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.667640 8550 raft_consensus.cc:2272] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.667650 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9e260030c0e6423cba37a2e3af1c6400 (table default.Table_17aa90b035264d2585988c8243238e7a [id=1072ac1d723c4628b4f71d0c203fa95f]) successfully deleted
I20251028 02:59:57.667868 8545 tablet_bootstrap.cc:492] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:57.667969 8545 ts_tablet_manager.cc:1403] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.000s
I20251028 02:59:57.668008 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 529b865389f94471a6849d17ff206e4c with delete_type TABLET_DATA_DELETED (replaced by tablet 240b7da6f46c4c5ab8087ab1dd6275c5) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.668036 8550 ts_tablet_manager.cc:1916] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.668175 8552 tablet_replica.cc:333] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.668319 8552 raft_consensus.cc:2243] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.668440 8552 raft_consensus.cc:2272] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.668432 8545 raft_consensus.cc:359] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.668546 8545 raft_consensus.cc:385] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.668592 8545 raft_consensus.cc:740] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
W20251028 02:59:57.668669 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 7b552f63895449aea79cbd8867cc063c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b552f63895449aea79cbd8867cc063c already in progress: creating tablet
I20251028 02:59:57.668711 8545 consensus_queue.cc:260] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:57.668804 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet c0f428e5c6fe43fc8f78761d868ddd09 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c0f428e5c6fe43fc8f78761d868ddd09 already in progress: creating tablet
W20251028 02:59:57.668926 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 8192667f14a44ec296d28efa6057a44d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8192667f14a44ec296d28efa6057a44d already in progress: creating tablet
I20251028 02:59:57.669014 8545 ts_tablet_manager.cc:1434] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 02:59:57.669025 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 39f0a9883b584083b542d078ea7c165b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 39f0a9883b584083b542d078ea7c165b already in progress: creating tablet
I20251028 02:59:57.669113 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 04d271c95c2847ca8b1fd2d55741117c (table default.Table_ad3466e031ae40059b8e3ec01c7ed5a9 [id=f00f5e8a1fef48e480906cbf79121e33]) successfully deleted
W20251028 02:59:57.669209 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f1d727c3dbef467da0aec436ffc393be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1d727c3dbef467da0aec436ffc393be already in progress: creating tablet
I20251028 02:59:57.669231 8545 tablet_bootstrap.cc:492] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:57.669358 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9583046f549d411a9e341560f428381c (table default.Table_8b7624342986474fbd0e31c86947f781 [id=530470c553154d76bf0df08ddfbf2fd0]) successfully deleted
W20251028 02:59:57.669448 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet f1d727c3dbef467da0aec436ffc393be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1d727c3dbef467da0aec436ffc393be already in progress: creating tablet
I20251028 02:59:57.670145 8544 tablet_bootstrap.cc:492] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.670218 8545 tablet_bootstrap.cc:654] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.670231 8544 ts_tablet_manager.cc:1403] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251028 02:59:57.671010 8544 raft_consensus.cc:359] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.671121 8544 raft_consensus.cc:385] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.671150 8544 raft_consensus.cc:740] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.671288 8544 consensus_queue.cc:260] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.671540 8550 ts_tablet_manager.cc:1929] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.671607 8550 log.cc:1199] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9583046f549d411a9e341560f428381c
I20251028 02:59:57.671666 8544 ts_tablet_manager.cc:1434] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251028 02:59:57.671813 8550 ts_tablet_manager.cc:1950] T 9583046f549d411a9e341560f428381c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.672410 8544 tablet_bootstrap.cc:492] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.673758 8544 tablet_bootstrap.cc:654] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.675292 8550 tablet_replica.cc:333] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.675557 8550 raft_consensus.cc:2243] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.675599 8546 tablet_bootstrap.cc:492] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.675695 8546 ts_tablet_manager.cc:1403] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.002s
I20251028 02:59:57.675714 8550 raft_consensus.cc:2272] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:57.676079 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ba64524142934af08658e434e7fb08ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba64524142934af08658e434e7fb08ed already in progress: creating tablet
I20251028 02:59:57.676241 8546 raft_consensus.cc:359] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.676326 8546 raft_consensus.cc:385] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.676345 8546 raft_consensus.cc:740] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.676349 8550 ts_tablet_manager.cc:1916] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.676453 8546 consensus_queue.cc:260] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.676718 8546 ts_tablet_manager.cc:1434] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:57.676901 8546 tablet_bootstrap.cc:492] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.677911 8546 tablet_bootstrap.cc:654] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.680290 8545 tablet_bootstrap.cc:492] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:57.680387 8545 ts_tablet_manager.cc:1403] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:57.680536 8552 ts_tablet_manager.cc:1916] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.680892 8545 raft_consensus.cc:359] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.680986 8545 raft_consensus.cc:385] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.681015 8545 raft_consensus.cc:740] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.681131 8545 consensus_queue.cc:260] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.681385 8545 ts_tablet_manager.cc:1434] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.681586 8545 tablet_bootstrap.cc:492] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:57.682507 8545 tablet_bootstrap.cc:654] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.682550 8550 ts_tablet_manager.cc:1929] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.682624 8550 log.cc:1199] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/529b865389f94471a6849d17ff206e4c
I20251028 02:59:57.682825 8552 ts_tablet_manager.cc:1929] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.682868 8550 ts_tablet_manager.cc:1950] T 529b865389f94471a6849d17ff206e4c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.682884 8552 log.cc:1199] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/529b865389f94471a6849d17ff206e4c
I20251028 02:59:57.683706 8550 tablet_replica.cc:333] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.683912 8550 raft_consensus.cc:2243] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.684080 8550 raft_consensus.cc:2272] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.684597 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 529b865389f94471a6849d17ff206e4c (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
I20251028 02:59:57.684826 8552 ts_tablet_manager.cc:1950] T 529b865389f94471a6849d17ff206e4c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.685464 8550 ts_tablet_manager.cc:1916] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.687255 9646 consensus_queue.cc:1048] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.688216 8550 ts_tablet_manager.cc:1929] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.689028 8550 log.cc:1199] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/35d5e3fa27ab46f9823b9cf4f7b3eb2a
I20251028 02:59:57.689344 8550 ts_tablet_manager.cc:1950] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.690158 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a (table default.Table_1b0dd83e47164c4da9d0b2311dfffc81 [id=6d26ce36ea3a479c9397a1e6ce995d68]) successfully deleted
I20251028 02:59:57.690564 8550 tablet_replica.cc:333] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.690742 8550 raft_consensus.cc:2243] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.690907 8550 raft_consensus.cc:2272] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.691711 8550 ts_tablet_manager.cc:1916] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.691905 9646 raft_consensus.cc:493] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.692006 9646 raft_consensus.cc:515] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.692425 9646 leader_election.cc:290] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.693220 8544 tablet_bootstrap.cc:492] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.693178 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d59acfb0e12643ebbb406dbda7bef29b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.693305 8544 ts_tablet_manager.cc:1403] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.001s
W20251028 02:59:57.693818 8083 leader_election.cc:343] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.694238 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d59acfb0e12643ebbb406dbda7bef29b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:57.694618 8083 leader_election.cc:343] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.694684 8083 leader_election.cc:304] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.694924 9646 raft_consensus.cc:2749] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.695701 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 529b865389f94471a6849d17ff206e4c (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
I20251028 02:59:57.696655 8545 tablet_bootstrap.cc:492] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:57.696761 8545 ts_tablet_manager.cc:1403] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 02:59:57.696909 8550 ts_tablet_manager.cc:1929] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.696971 8550 log.cc:1199] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/6dfc5240893546d7a6a5397cd08044dd
I20251028 02:59:57.696964 8546 tablet_bootstrap.cc:492] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.697050 8546 ts_tablet_manager.cc:1403] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.002s
I20251028 02:59:57.697185 8550 ts_tablet_manager.cc:1950] T 6dfc5240893546d7a6a5397cd08044dd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.697309 8545 raft_consensus.cc:359] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.697405 8545 raft_consensus.cc:385] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.697438 8545 raft_consensus.cc:740] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.697520 8546 raft_consensus.cc:359] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.697613 8546 raft_consensus.cc:385] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.697563 8545 consensus_queue.cc:260] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.697643 8546 raft_consensus.cc:740] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.697801 8550 tablet_replica.cc:333] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.697760 8546 consensus_queue.cc:260] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.697892 8545 ts_tablet_manager.cc:1434] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.697925 8550 raft_consensus.cc:2243] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.698066 8546 ts_tablet_manager.cc:1434] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:57.698107 8545 tablet_bootstrap.cc:492] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:57.698247 8546 tablet_bootstrap.cc:492] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.698947 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.699102 8545 tablet_bootstrap.cc:654] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.699695 8550 raft_consensus.cc:2272] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.700320 8550 ts_tablet_manager.cc:1916] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.701550 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 6dfc5240893546d7a6a5397cd08044dd (table default.Table_c20f674e0d9a4178a66f16d31bc99638 [id=49d0cfc3409f4e2d84b5e1b2634193e5]) successfully deleted
I20251028 02:59:57.703873 8550 ts_tablet_manager.cc:1929] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.704017 8550 log.cc:1199] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/f6ac7b5fff0848cebf710ba3ed50def4
I20251028 02:59:57.704411 8552 tablet_replica.cc:333] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.704416 8544 raft_consensus.cc:359] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.704509 8544 raft_consensus.cc:385] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.704540 8544 raft_consensus.cc:740] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.704574 8552 raft_consensus.cc:2243] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.704684 8552 raft_consensus.cc:2272] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.704655 8544 consensus_queue.cc:260] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.704937 8544 ts_tablet_manager.cc:1434] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20251028 02:59:57.707579 8544 tablet_bootstrap.cc:492] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.707700 7963 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 81b521c9f57844bb9873f6b26b5cf661 (table default.Table_fd3daefb8df048e7bea5949c30da1e1f [id=5f82b9596e5842deb4d45e19c91629bb]). Expected version 1 got 0
I20251028 02:59:57.711756 8112 tablet_service.cc:1505] Processing CreateTablet for tablet f03b90d615ab435f9a244504d8d01660 (DEFAULT_TABLE table=default.Table_206be08f6668466db643e3bf2fff50ce [id=3324d556b6384d519585376a317740e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.711977 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 07509e5c36d9440fa9fe29c251931c63 (DEFAULT_TABLE table=default.Table_feec1045abaf4ffb9ef0e698d450a0c7 [id=68489b66f73f44ddb7b522b84fad4eea]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.712088 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f03b90d615ab435f9a244504d8d01660. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.712260 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07509e5c36d9440fa9fe29c251931c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.713653 8544 tablet_bootstrap.cc:654] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.714632 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 746eae30ca284758a6e3ed8ad80c2cdc (DEFAULT_TABLE table=default.Table_2681fbe951fb4a88bbf4c192aac594d1 [id=9dfdc263b81f4531947c71446c937a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.714844 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746eae30ca284758a6e3ed8ad80c2cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.716568 8552 ts_tablet_manager.cc:1916] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.717695 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet d59acfb0e12643ebbb406dbda7bef29b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.718628 8562 consensus_queue.cc:1048] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.721010 9639 raft_consensus.cc:493] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.721115 9639 raft_consensus.cc:515] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.721510 9639 leader_election.cc:290] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.721902 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49b46c13cb4e46388ed0a69d2ced9b42" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.723380 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "49b46c13cb4e46388ed0a69d2ced9b42" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:57.725292 8354 leader_election.cc:304] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:57.725453 9639 raft_consensus.cc:2749] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.727059 8546 tablet_bootstrap.cc:654] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.728904 8552 ts_tablet_manager.cc:1929] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.729589 8552 log.cc:1199] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/35d5e3fa27ab46f9823b9cf4f7b3eb2a
I20251028 02:59:57.730050 8552 ts_tablet_manager.cc:1950] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.730022 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 2d0e84daebfe481c9df8459e33b46e16 (DEFAULT_TABLE table=default.Table_07cff4b56d204f6483cbe27f181cc80a [id=79728531deaf4c47b84d78b2829670b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.730309 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d0e84daebfe481c9df8459e33b46e16. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.731333 8127 tablet_service.cc:1505] Processing CreateTablet for tablet a0943be517fd49e199fd060782a22c31 (DEFAULT_TABLE table=default.Table_fcb100729f5b48b990a9c4d2d9b9fcf2 [id=0381bc31864343c19a461fe49735d58a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.731505 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 39807cf770b045cca6f66211831edc37 (DEFAULT_TABLE table=default.Table_3249352b73e544a1afa41660422fe744 [id=4e064d4c691247a89bbece7f3ac6076d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.731590 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0943be517fd49e199fd060782a22c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.731750 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39807cf770b045cca6f66211831edc37. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.731942 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 127e6316879e429998799ba17f0b17d5 (DEFAULT_TABLE table=default.Table_7c8e2218f01e4adbbff3b276fe5345d2 [id=050dc39ec88e457f9a4d29cdc14d9ccb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.732074 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 07509e5c36d9440fa9fe29c251931c63 (DEFAULT_TABLE table=default.Table_feec1045abaf4ffb9ef0e698d450a0c7 [id=68489b66f73f44ddb7b522b84fad4eea]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.732189 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127e6316879e429998799ba17f0b17d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.732312 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07509e5c36d9440fa9fe29c251931c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.732555 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 4eaa472029a94052abb49241b7622d0f (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.732745 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eaa472029a94052abb49241b7622d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.733954 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 7ee583e318174f34b162c2a104a0a3a9 (DEFAULT_TABLE table=default.Table_b6f5cef4fc824b09ab85a48c5bd021a7 [id=fbb760e3c8d24c6aaf36e3fbc2a4ae6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.734232 8116 tablet_service.cc:1505] Processing CreateTablet for tablet de0ba9623bbc4d09abc7a3571046cdce (DEFAULT_TABLE table=default.Table_78fb1bab55774710bbc8014a9085fb0d [id=43844eeb88004b968a1996a1e77ab94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.734493 8117 tablet_service.cc:1505] Processing CreateTablet for tablet f898e9072d7d4697aee6a981cf6188b4 (DEFAULT_TABLE table=default.Table_cc716ae9a4cd4b939b2e461342f03331 [id=7375e54ec4fc4747bf8e8db863ac69ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.734763 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ee583e318174f34b162c2a104a0a3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.734731 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet f6ac7b5fff0848cebf710ba3ed50def4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f90e3c3524e4498aa6c4bfc5304d44b) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.734710 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 02f1a2d489d74e33ab991611fd55966d (DEFAULT_TABLE table=default.Table_6eabdbe1000e4fb6b82c9425cc2e0edd [id=f2b49da1d9f0421f84ff98bc5c05462f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.734957 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02f1a2d489d74e33ab991611fd55966d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.735113 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f898e9072d7d4697aee6a981cf6188b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.735708 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 07509e5c36d9440fa9fe29c251931c63 (DEFAULT_TABLE table=default.Table_feec1045abaf4ffb9ef0e698d450a0c7 [id=68489b66f73f44ddb7b522b84fad4eea]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.735903 8386 tablet_service.cc:1505] Processing CreateTablet for tablet f03b90d615ab435f9a244504d8d01660 (DEFAULT_TABLE table=default.Table_206be08f6668466db643e3bf2fff50ce [id=3324d556b6384d519585376a317740e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.735997 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07509e5c36d9440fa9fe29c251931c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.736157 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f03b90d615ab435f9a244504d8d01660. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.737076 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0ba9623bbc4d09abc7a3571046cdce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.738301 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 746eae30ca284758a6e3ed8ad80c2cdc (DEFAULT_TABLE table=default.Table_2681fbe951fb4a88bbf4c192aac594d1 [id=9dfdc263b81f4531947c71446c937a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.738298 8263 tablet_service.cc:1505] Processing CreateTablet for tablet f03b90d615ab435f9a244504d8d01660 (DEFAULT_TABLE table=default.Table_206be08f6668466db643e3bf2fff50ce [id=3324d556b6384d519585376a317740e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.738512 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746eae30ca284758a6e3ed8ad80c2cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.738515 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f03b90d615ab435f9a244504d8d01660. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.739758 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 746eae30ca284758a6e3ed8ad80c2cdc (DEFAULT_TABLE table=default.Table_2681fbe951fb4a88bbf4c192aac594d1 [id=9dfdc263b81f4531947c71446c937a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.739986 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746eae30ca284758a6e3ed8ad80c2cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.740077 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 2d0e84daebfe481c9df8459e33b46e16 (DEFAULT_TABLE table=default.Table_07cff4b56d204f6483cbe27f181cc80a [id=79728531deaf4c47b84d78b2829670b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.740216 8552 tablet_replica.cc:333] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.740291 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d0e84daebfe481c9df8459e33b46e16. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.740372 8552 raft_consensus.cc:2243] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.740485 8552 raft_consensus.cc:2272] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.741232 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 7e0e038d33bf45f693da251cd560ae2a (DEFAULT_TABLE table=default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.741921 8258 tablet_service.cc:1505] Processing CreateTablet for tablet de0ba9623bbc4d09abc7a3571046cdce (DEFAULT_TABLE table=default.Table_78fb1bab55774710bbc8014a9085fb0d [id=43844eeb88004b968a1996a1e77ab94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.742017 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet cbecb6472dc54989beca91ca7bcb0b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.742151 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0ba9623bbc4d09abc7a3571046cdce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.742273 8384 tablet_service.cc:1505] Processing CreateTablet for tablet 39807cf770b045cca6f66211831edc37 (DEFAULT_TABLE table=default.Table_3249352b73e544a1afa41660422fe744 [id=4e064d4c691247a89bbece7f3ac6076d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.742291 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 2d0e84daebfe481c9df8459e33b46e16 (DEFAULT_TABLE table=default.Table_07cff4b56d204f6483cbe27f181cc80a [id=79728531deaf4c47b84d78b2829670b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.742483 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d0e84daebfe481c9df8459e33b46e16. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.742578 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a (table default.Table_1b0dd83e47164c4da9d0b2311dfffc81 [id=6d26ce36ea3a479c9397a1e6ce995d68]) successfully deleted
I20251028 02:59:57.742617 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 7e0e038d33bf45f693da251cd560ae2a (DEFAULT_TABLE table=default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.742810 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e0e038d33bf45f693da251cd560ae2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.743497 8253 tablet_service.cc:1505] Processing CreateTablet for tablet 7ee583e318174f34b162c2a104a0a3a9 (DEFAULT_TABLE table=default.Table_b6f5cef4fc824b09ab85a48c5bd021a7 [id=fbb760e3c8d24c6aaf36e3fbc2a4ae6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.743707 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ee583e318174f34b162c2a104a0a3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.744340 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4675220a671f4306b8f46597ab3ca721"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.744517 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 39807cf770b045cca6f66211831edc37 (DEFAULT_TABLE table=default.Table_3249352b73e544a1afa41660422fe744 [id=4e064d4c691247a89bbece7f3ac6076d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.744736 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39807cf770b045cca6f66211831edc37. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:57.744740 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4675220a671f4306b8f46597ab3ca721 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.745482 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1af3c83bed1b4fd2bbc9f6a0b0716489"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.745669 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 7ee583e318174f34b162c2a104a0a3a9 (DEFAULT_TABLE table=default.Table_b6f5cef4fc824b09ab85a48c5bd021a7 [id=fbb760e3c8d24c6aaf36e3fbc2a4ae6c]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:57.745764 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1af3c83bed1b4fd2bbc9f6a0b0716489 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.746042 8256 tablet_service.cc:1505] Processing CreateTablet for tablet a0943be517fd49e199fd060782a22c31 (DEFAULT_TABLE table=default.Table_fcb100729f5b48b990a9c4d2d9b9fcf2 [id=0381bc31864343c19a461fe49735d58a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.746249 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0943be517fd49e199fd060782a22c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.746517 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7a237cac752f41ea9761c1845b56171d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.746785 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a237cac752f41ea9761c1845b56171d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.746827 8398 tablet_service.cc:1505] Processing CreateTablet for tablet a0943be517fd49e199fd060782a22c31 (DEFAULT_TABLE table=default.Table_fcb100729f5b48b990a9c4d2d9b9fcf2 [id=0381bc31864343c19a461fe49735d58a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.747238 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 127e6316879e429998799ba17f0b17d5 (DEFAULT_TABLE table=default.Table_7c8e2218f01e4adbbff3b276fe5345d2 [id=050dc39ec88e457f9a4d29cdc14d9ccb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.747471 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127e6316879e429998799ba17f0b17d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.747584 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ee1375b2eca6432291e041e9aaf7bf3a" } new_table_name: "default.Table_0d32d1fdb9bb4294a8b7054819d111e5" modify_external_catalogs: true
I20251028 02:59:57.748291 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 127e6316879e429998799ba17f0b17d5 (DEFAULT_TABLE table=default.Table_7c8e2218f01e4adbbff3b276fe5345d2 [id=050dc39ec88e457f9a4d29cdc14d9ccb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.748323 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6f0cc2934b86479083615ebdb7ead649" } modify_external_catalogs: true
I20251028 02:59:57.748409 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6f0cc2934b86479083615ebdb7ead649" } modify_external_catalogs: true
I20251028 02:59:57.748692 8400 tablet_service.cc:1505] Processing CreateTablet for tablet de0ba9623bbc4d09abc7a3571046cdce (DEFAULT_TABLE table=default.Table_78fb1bab55774710bbc8014a9085fb0d [id=43844eeb88004b968a1996a1e77ab94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.749507 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 4eaa472029a94052abb49241b7622d0f (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.749713 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eaa472029a94052abb49241b7622d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.750344 8387 tablet_service.cc:1505] Processing CreateTablet for tablet f898e9072d7d4697aee6a981cf6188b4 (DEFAULT_TABLE table=default.Table_cc716ae9a4cd4b939b2e461342f03331 [id=7375e54ec4fc4747bf8e8db863ac69ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.750689 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 02f1a2d489d74e33ab991611fd55966d (DEFAULT_TABLE table=default.Table_6eabdbe1000e4fb6b82c9425cc2e0edd [id=f2b49da1d9f0421f84ff98bc5c05462f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.750945 8562 raft_consensus.cc:493] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.751032 8562 raft_consensus.cc:515] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.751606 8545 tablet_bootstrap.cc:492] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:57.751736 8551 tablet_replica.cc:333] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.751892 8551 raft_consensus.cc:2243] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.751899 8260 tablet_service.cc:1505] Processing CreateTablet for tablet f898e9072d7d4697aee6a981cf6188b4 (DEFAULT_TABLE table=default.Table_cc716ae9a4cd4b939b2e461342f03331 [id=7375e54ec4fc4747bf8e8db863ac69ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.751999 8551 raft_consensus.cc:2272] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.751731 8545 ts_tablet_manager.cc:1403] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.054s user 0.001s sys 0.001s
I20251028 02:59:57.752128 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f898e9072d7d4697aee6a981cf6188b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.752844 8545 raft_consensus.cc:359] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.753013 8545 raft_consensus.cc:385] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.753083 8545 raft_consensus.cc:740] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.753278 8545 consensus_queue.cc:260] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.753491 8562 leader_election.cc:290] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.754170 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09d65eef81b2421c92678015b6da4fbb" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.754168 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_dbd811be59344e8587fff44779930d85" } new_table_name: "default.Table_ba93ba86d1664f40b4e919124f42b551" modify_external_catalogs: true
I20251028 02:59:57.755263 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1d2bfae88392496383bd49d6f3dd832f" } new_table_name: "default.Table_11f1aff812d14b0f8b0004ae5438d7e0" modify_external_catalogs: true
I20251028 02:59:57.755563 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_479f49bd28d1417a94221c750005fe3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.755849 8083 leader_election.cc:343] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:57.755942 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_479f49bd28d1417a94221c750005fe3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.757293 8546 tablet_bootstrap.cc:492] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.757385 8546 ts_tablet_manager.cc:1403] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.059s user 0.000s sys 0.002s
I20251028 02:59:57.757875 8546 raft_consensus.cc:359] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.757941 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09d65eef81b2421c92678015b6da4fbb" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.757992 8546 raft_consensus.cc:385] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.758023 8546 raft_consensus.cc:740] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.758126 8546 consensus_queue.cc:260] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.758342 8545 ts_tablet_manager.cc:1434] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20251028 02:59:57.758419 8546 ts_tablet_manager.cc:1434] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:57.759241 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e1a6fdd96e72488ea4a3f242f5d52d27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.759356 8545 tablet_bootstrap.cc:492] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:57.759662 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1a6fdd96e72488ea4a3f242f5d52d27 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.760891 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_cd4df8be533e472bafeb04d0aa635832"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.761108 8083 leader_election.cc:343] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.761189 8083 leader_election.cc:304] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
W20251028 02:59:57.761247 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd4df8be533e472bafeb04d0aa635832 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.761899 8562 raft_consensus.cc:2749] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.762707 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 4eaa472029a94052abb49241b7622d0f (DEFAULT_TABLE table=default.Table_9090533beeb54c4cb4a11365200d3b4e [id=5581f43ea3854dbeba821353ffce6ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.763779 8551 ts_tablet_manager.cc:1916] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.763960 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_196d7bd17aed4ca7909e5ab6e7653130"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.764333 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_196d7bd17aed4ca7909e5ab6e7653130 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.765838 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e0e038d33bf45f693da251cd560ae2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.766115 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eaa472029a94052abb49241b7622d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.766456 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39807cf770b045cca6f66211831edc37. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.767408 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02f1a2d489d74e33ab991611fd55966d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.768337 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0943be517fd49e199fd060782a22c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.768433 8551 ts_tablet_manager.cc:1929] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.768497 8551 log.cc:1199] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/cbecb6472dc54989beca91ca7bcb0b57
I20251028 02:59:57.768702 8551 ts_tablet_manager.cc:1950] T cbecb6472dc54989beca91ca7bcb0b57 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.769366 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f898e9072d7d4697aee6a981cf6188b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.770411 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0ba9623bbc4d09abc7a3571046cdce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.771406 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ee583e318174f34b162c2a104a0a3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.771643 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet cbecb6472dc54989beca91ca7bcb0b57 (table default.Table_3292bf8a27264e8491009af851138333 [id=d317a63dea224c45b4dfd1a205ae4c68]) successfully deleted
I20251028 02:59:57.773005 8545 tablet_bootstrap.cc:654] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.773226 8546 tablet_bootstrap.cc:492] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.773293 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 127e6316879e429998799ba17f0b17d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.774178 8546 tablet_bootstrap.cc:654] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.777858 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 02f1a2d489d74e33ab991611fd55966d (DEFAULT_TABLE table=default.Table_6eabdbe1000e4fb6b82c9425cc2e0edd [id=f2b49da1d9f0421f84ff98bc5c05462f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.778147 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02f1a2d489d74e33ab991611fd55966d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.779832 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 7e0e038d33bf45f693da251cd560ae2a (DEFAULT_TABLE table=default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.780128 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e0e038d33bf45f693da251cd560ae2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.781075 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ffe824a618714776bfab38679c9565e0
W20251028 02:59:57.781318 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2fc2faa9c4f04fe3b931ce8a2808d738 consensus state has no leader
I20251028 02:59:57.781327 8552 ts_tablet_manager.cc:1916] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.782655 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a290b8114844aa89b33c75fffc8e1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.783085 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet ffe824a618714776bfab38679c9565e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.783334 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet ffe824a618714776bfab38679c9565e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.784677 8551 tablet_replica.cc:333] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.784814 8551 raft_consensus.cc:2243] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.784910 8551 raft_consensus.cc:2272] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.785514 8551 ts_tablet_manager.cc:1916] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.785921 8552 ts_tablet_manager.cc:1929] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.786087 8552 log.cc:1199] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/f6ac7b5fff0848cebf710ba3ed50def4
I20251028 02:59:57.786501 8552 ts_tablet_manager.cc:1950] T f6ac7b5fff0848cebf710ba3ed50def4 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.786918 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet ffe824a618714776bfab38679c9565e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.787238 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet f6ac7b5fff0848cebf710ba3ed50def4 (table default.Table_42f89d7fccbf4c738293e9796be235a0 [id=6f3e689a22ae4160ae7537d955c06a90]) successfully deleted
I20251028 02:59:57.787815 8551 ts_tablet_manager.cc:1929] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.787878 8551 log.cc:1199] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9a290b8114844aa89b33c75fffc8e1e7
W20251028 02:59:57.787945 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ffe824a618714776bfab38679c9565e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffe824a618714776bfab38679c9565e0 already in progress: creating tablet
I20251028 02:59:57.788074 8551 ts_tablet_manager.cc:1950] T 9a290b8114844aa89b33c75fffc8e1e7 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.788690 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9a290b8114844aa89b33c75fffc8e1e7 (table default.Table_13cf93a2eb1f41379907b8639554e44a [id=bdc2d57238f84580b7f15c6fde60f921]) successfully deleted
W20251028 02:59:57.789111 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ffe824a618714776bfab38679c9565e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffe824a618714776bfab38679c9565e0 already in progress: creating tablet
I20251028 02:59:57.789309 8550 ts_tablet_manager.cc:1950] T f6ac7b5fff0848cebf710ba3ed50def4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.789968 8550 tablet_replica.cc:333] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.790114 8545 tablet_bootstrap.cc:492] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:57.790133 8550 raft_consensus.cc:2243] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.790254 8550 raft_consensus.cc:2272] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.790854 8545 ts_tablet_manager.cc:1403] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20251028 02:59:57.790903 8550 ts_tablet_manager.cc:1916] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.792003 8545 raft_consensus.cc:359] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.792122 8545 raft_consensus.cc:385] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.792156 8545 raft_consensus.cc:740] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.792325 8545 consensus_queue.cc:260] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.793118 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet bd50e7d4e28541de9799833c53fe7c02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.793465 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet f6ac7b5fff0848cebf710ba3ed50def4 (table default.Table_42f89d7fccbf4c738293e9796be235a0 [id=6f3e689a22ae4160ae7537d955c06a90]) successfully deleted
I20251028 02:59:57.793520 8550 ts_tablet_manager.cc:1929] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:57.793586 8550 log.cc:1199] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/b6d868c81a7e402ba9305bb2aa90625b
I20251028 02:59:57.794957 8546 tablet_bootstrap.cc:492] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.795064 8546 ts_tablet_manager.cc:1403] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.001s
I20251028 02:59:57.795568 8546 raft_consensus.cc:359] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.795679 8546 raft_consensus.cc:385] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.795710 8546 raft_consensus.cc:740] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.795966 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]). Expected version 2 got 0
I20251028 02:59:57.796001 8546 consensus_queue.cc:260] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.796321 8546 ts_tablet_manager.cc:1434] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.796554 8546 tablet_bootstrap.cc:492] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.797434 8546 tablet_bootstrap.cc:654] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.799360 8550 ts_tablet_manager.cc:1950] T b6d868c81a7e402ba9305bb2aa90625b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.799574 8544 tablet_bootstrap.cc:492] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.799659 8544 ts_tablet_manager.cc:1403] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.092s user 0.002s sys 0.001s
I20251028 02:59:57.800140 8544 raft_consensus.cc:359] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.800266 8544 raft_consensus.cc:385] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.800307 8544 raft_consensus.cc:740] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.800415 8544 consensus_queue.cc:260] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.800714 8544 ts_tablet_manager.cc:1434] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.800738 8550 tablet_replica.cc:333] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.800709 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet b6d868c81a7e402ba9305bb2aa90625b (table default.Table_19f5792cad7f4b2fa7f0446f6c3c7bb0 [id=da79674aa8cd468e84114457ee46766e]) successfully deleted
I20251028 02:59:57.800854 8550 raft_consensus.cc:2243] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:57.800877 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 07d3e6c640434652a010942402c54716 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07d3e6c640434652a010942402c54716 already in progress: creating tablet
I20251028 02:59:57.800917 8544 tablet_bootstrap.cc:492] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.800946 8550 raft_consensus.cc:2272] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:57.800994 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a already in progress: creating tablet
I20251028 02:59:57.801901 8544 tablet_bootstrap.cc:654] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:57.802528 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2fec0adb79d04469b90c1ea2a5e9ba5b already in progress: creating tablet
I20251028 02:59:57.802884 9639 raft_consensus.cc:493] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.802980 9639 raft_consensus.cc:515] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:57.803131 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d60fc654f6af44b290d46235082a4113 consensus state has no leader
I20251028 02:59:57.803454 9639 leader_election.cc:290] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.804488 8552 tablet_replica.cc:333] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.804477 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5428db004de74f7492c9b8999e53020e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:57.804617 8145 raft_consensus.cc:2393] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7e4cf694f71438298dd32a346abcf75 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:57.804653 8552 raft_consensus.cc:2243] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.804922 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5428db004de74f7492c9b8999e53020e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.805378 8552 raft_consensus.cc:2272] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.805974 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet c0f428e5c6fe43fc8f78761d868ddd09 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.806010 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet f08a6e30211a4d59934c36c7a26ac983 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.806483 8356 leader_election.cc:304] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:57.807355 9639 raft_consensus.cc:3060] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.808079 8419 raft_consensus.cc:3060] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.808383 8550 ts_tablet_manager.cc:1916] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.808537 9639 raft_consensus.cc:2749] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.809121 8419 raft_consensus.cc:1275] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:57.809384 8145 raft_consensus.cc:1275] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:57.809536 9595 consensus_queue.cc:1048] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.810493 8550 ts_tablet_manager.cc:1929] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.810555 8550 log.cc:1199] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/7b552f63895449aea79cbd8867cc063c
I20251028 02:59:57.810726 8550 ts_tablet_manager.cc:1950] T 7b552f63895449aea79cbd8867cc063c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.810765 9595 consensus_queue.cc:1048] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.815050 8546 tablet_bootstrap.cc:492] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.815141 8544 tablet_bootstrap.cc:492] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.815189 8546 ts_tablet_manager.cc:1403] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20251028 02:59:57.815243 8544 ts_tablet_manager.cc:1403] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20251028 02:59:57.815712 8546 raft_consensus.cc:359] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.815712 8544 raft_consensus.cc:359] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.815821 8546 raft_consensus.cc:385] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.815821 8544 raft_consensus.cc:385] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.815857 8546 raft_consensus.cc:740] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.815873 8544 raft_consensus.cc:740] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.815990 8544 consensus_queue.cc:260] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.816270 8544 ts_tablet_manager.cc:1434] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.816458 8545 ts_tablet_manager.cc:1434] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.026s user 0.001s sys 0.000s
I20251028 02:59:57.816706 8545 tablet_bootstrap.cc:492] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:57.816923 8546 consensus_queue.cc:260] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.817555 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a19abcac1bc24d7aa75a6032b0689b1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.817710 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8c6b8c4cb0e84fb88487023453581cec" } new_table_name: "default.Table_79ced5936022425c91473064b87986d3" modify_external_catalogs: true
W20251028 02:59:57.817996 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a19abcac1bc24d7aa75a6032b0689b1b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.818740 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c175a83383134da183a6ca9e861ea2ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.819067 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c175a83383134da183a6ca9e861ea2ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.819937 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e1a6fdd96e72488ea4a3f242f5d52d27" } new_table_name: "default.Table_abcc95f588d04951b016ffd9ea9800c5" modify_external_catalogs: true
I20251028 02:59:57.822017 8545 tablet_bootstrap.cc:654] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.823868 8544 tablet_bootstrap.cc:492] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.824694 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet fcab55fd49da41f7a95d46d76a8dac8b (table default.Table_11f1aff812d14b0f8b0004ae5438d7e0 [id=86e0c4d129fc42f1998b2b0f2dbc2135]). Expected version 1 got 0
I20251028 02:59:57.824957 8544 tablet_bootstrap.cc:654] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.826560 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 7b552f63895449aea79cbd8867cc063c (table default.Table_8c2ee567a0b74836997f0d500266c256 [id=efaa3797242b4a238c7aac2d7427451c]) successfully deleted
I20251028 02:59:57.826618 8546 ts_tablet_manager.cc:1434] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
W20251028 02:59:57.826733 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f1d727c3dbef467da0aec436ffc393be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1d727c3dbef467da0aec436ffc393be already in progress: creating tablet
I20251028 02:59:57.829711 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]). Expected version 2 got 0
I20251028 02:59:57.835762 8550 tablet_replica.cc:333] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.835939 8550 raft_consensus.cc:2243] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.835990 8546 tablet_bootstrap.cc:492] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.836056 8550 raft_consensus.cc:2272] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.836575 8550 ts_tablet_manager.cc:1916] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.834635 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 8483cbc3099644d2b6f7ff52869e21ea from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:57.834944 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8720ea8c68904d57a3fc5c0e94d338ca consensus state has no leader
W20251028 02:59:57.835407 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f79e2add439c4d36af2e84eacbcdbc8f has no consensus state
W20251028 02:59:57.836390 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 39f0a9883b584083b542d078ea7c165b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 39f0a9883b584083b542d078ea7c165b already in progress: creating tablet
I20251028 02:59:57.839429 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 874b6660d86b48ebad03ecc54588da28 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d3e422c7ae64b25960714ecc926046a) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.841867 8546 tablet_bootstrap.cc:654] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.842242 8552 ts_tablet_manager.cc:1916] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.844167 8544 tablet_bootstrap.cc:492] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.844259 8544 ts_tablet_manager.cc:1403] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251028 02:59:57.844704 8544 raft_consensus.cc:359] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.844802 8544 raft_consensus.cc:385] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.844830 8544 raft_consensus.cc:740] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.844941 8544 consensus_queue.cc:260] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.845206 8544 ts_tablet_manager.cc:1434] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.845394 8544 tablet_bootstrap.cc:492] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.846350 8544 tablet_bootstrap.cc:654] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.846659 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 02695b18936d4f928cd55f9b76b90b2e from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.846863 8550 ts_tablet_manager.cc:1929] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.846931 8550 log.cc:1199] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c0f428e5c6fe43fc8f78761d868ddd09
I20251028 02:59:57.848728 8550 ts_tablet_manager.cc:1950] T c0f428e5c6fe43fc8f78761d868ddd09 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.848950 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8249c2c329814e99a3636633a900f8bd" } modify_external_catalogs: true
I20251028 02:59:57.851182 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8249c2c329814e99a3636633a900f8bd" } modify_external_catalogs: true
I20251028 02:59:57.849483 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_929b382a93794515a84b4f6356fb86b2" } modify_external_catalogs: true
I20251028 02:59:57.851913 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_929b382a93794515a84b4f6356fb86b2" } modify_external_catalogs: true
I20251028 02:59:57.849558 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4045ef6d27fd4f2fbfb8975411994305" } modify_external_catalogs: true
I20251028 02:59:57.852185 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4045ef6d27fd4f2fbfb8975411994305" } modify_external_catalogs: true
I20251028 02:59:57.848456 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7c8e2218f01e4adbbff3b276fe5345d2" } modify_external_catalogs: true
I20251028 02:59:57.853631 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7c8e2218f01e4adbbff3b276fe5345d2" } modify_external_catalogs: true
I20251028 02:59:57.854403 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4673cd2bd8f2439c94aac247f1ce8f12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.854858 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4673cd2bd8f2439c94aac247f1ce8f12 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.855104 9551 tablet.cc:1722] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.855455 8545 tablet_bootstrap.cc:492] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:57.855552 8545 ts_tablet_manager.cc:1403] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20251028 02:59:57.856073 8545 raft_consensus.cc:359] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.856173 8545 raft_consensus.cc:385] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.856202 8545 raft_consensus.cc:740] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.856375 8545 consensus_queue.cc:260] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.856700 8545 ts_tablet_manager.cc:1434] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.856958 8545 tablet_bootstrap.cc:492] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:57.858093 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet c0f428e5c6fe43fc8f78761d868ddd09 (table default.Table_594bc3babcbf48e4b5d9490f18936563 [id=0ee1306a57fd419497c97f844e58157e]) successfully deleted
W20251028 02:59:57.858629 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 8192667f14a44ec296d28efa6057a44d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8192667f14a44ec296d28efa6057a44d already in progress: creating tablet
I20251028 02:59:57.858795 8562 raft_consensus.cc:493] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.858845 8545 tablet_bootstrap.cc:654] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.858898 8562 raft_consensus.cc:515] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.859344 8562 leader_election.cc:290] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.859684 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b46554b0df894beb8141126aa7ee7b8e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.860196 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b46554b0df894beb8141126aa7ee7b8e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:57.860430 8083 leader_election.cc:343] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:57.860724 8083 leader_election.cc:343] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.861400 8083 leader_election.cc:304] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.861600 8562 raft_consensus.cc:2749] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.863557 8550 tablet_replica.cc:333] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.863587 8546 tablet_bootstrap.cc:492] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.863688 8546 ts_tablet_manager.cc:1403] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.002s
I20251028 02:59:57.864215 8546 raft_consensus.cc:359] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.865046 9529 tablet.cc:1722] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.864324 8546 raft_consensus.cc:385] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.865373 8546 raft_consensus.cc:740] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.865504 8546 consensus_queue.cc:260] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.865810 8546 ts_tablet_manager.cc:1434] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 02:59:57.865986 8546 tablet_bootstrap.cc:492] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.866220 9595 raft_consensus.cc:493] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.866323 9595 raft_consensus.cc:515] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.866693 9595 leader_election.cc:290] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.867012 8546 tablet_bootstrap.cc:654] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.867153 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "874b6660d86b48ebad03ecc54588da28" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:57.867525 8220 leader_election.cc:343] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.869546 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 06302d2cb5554805b85916741ea10f0c from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.870256 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "874b6660d86b48ebad03ecc54588da28" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:57.870383 8145 raft_consensus.cc:2468] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:57.870573 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 429b9cd6b775436a865afd64ec5eae4f from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.871467 8218 leader_election.cc:304] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.871724 9685 raft_consensus.cc:2804] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:57.871783 9685 raft_consensus.cc:493] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:57.871821 9685 raft_consensus.cc:3060] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.872534 8562 raft_consensus.cc:493] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.872613 8562 raft_consensus.cc:515] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.872948 8562 leader_election.cc:290] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.872996 9685 raft_consensus.cc:515] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.873351 9685 leader_election.cc:290] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.873593 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f08a6e30211a4d59934c36c7a26ac983" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.873778 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f08a6e30211a4d59934c36c7a26ac983" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.874212 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "874b6660d86b48ebad03ecc54588da28" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
W20251028 02:59:57.874274 8083 leader_election.cc:343] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:57.874402 8083 leader_election.cc:343] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.874459 8083 leader_election.cc:304] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.874583 8562 raft_consensus.cc:2749] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251028 02:59:57.874614 8220 leader_election.cc:343] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.874922 8550 raft_consensus.cc:2243] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.875089 8550 raft_consensus.cc:2272] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.875552 8420 raft_consensus.cc:3060] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.876498 8544 tablet_bootstrap.cc:492] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.876579 8544 ts_tablet_manager.cc:1403] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.000s
I20251028 02:59:57.877054 8544 raft_consensus.cc:359] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.877185 8544 raft_consensus.cc:385] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.877223 8544 raft_consensus.cc:740] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.877331 8544 consensus_queue.cc:260] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.877614 8544 ts_tablet_manager.cc:1434] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.877785 8544 tablet_bootstrap.cc:492] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.878787 8544 tablet_bootstrap.cc:654] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.880448 8550 ts_tablet_manager.cc:1916] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.880574 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "874b6660d86b48ebad03ecc54588da28" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:57.880677 8145 raft_consensus.cc:3060] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.881035 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 9583046f549d411a9e341560f428381c with delete_type TABLET_DATA_DELETED (replaced by tablet ba64524142934af08658e434e7fb08ed) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.881211 8551 tablet_replica.cc:333] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.881332 8551 raft_consensus.cc:2243] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.881428 8551 raft_consensus.cc:2272] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.881654 8145 raft_consensus.cc:2468] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:57.881738 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 429b9cd6b775436a865afd64ec5eae4f
I20251028 02:59:57.882016 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 429b9cd6b775436a865afd64ec5eae4f deleted and replaced by tablet 69c9f986f7004ee6a0b6bba624ae1f8d
I20251028 02:59:57.882128 8218 leader_election.cc:304] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.882519 9685 raft_consensus.cc:2804] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:57.882616 9685 raft_consensus.cc:697] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:57.882752 9685 consensus_queue.cc:237] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.882870 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 06302d2cb5554805b85916741ea10f0c
I20251028 02:59:57.883116 7965 catalog_manager.cc:6719] ReplaceTablet: tablet 06302d2cb5554805b85916741ea10f0c deleted and replaced by tablet b4ef27777bc047b886a1be0d4622dcf8
I20251028 02:59:57.882526 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 429b9cd6b775436a865afd64ec5eae4f with delete_type TABLET_DATA_DELETED (replaced by tablet 69c9f986f7004ee6a0b6bba624ae1f8d) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.883559 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 06302d2cb5554805b85916741ea10f0c with delete_type TABLET_DATA_DELETED (replaced by tablet b4ef27777bc047b886a1be0d4622dcf8) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.883833 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 06302d2cb5554805b85916741ea10f0c with delete_type TABLET_DATA_DELETED (replaced by tablet b4ef27777bc047b886a1be0d4622dcf8) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.884230 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 429b9cd6b775436a865afd64ec5eae4f with delete_type TABLET_DATA_DELETED (replaced by tablet 69c9f986f7004ee6a0b6bba624ae1f8d) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.884567 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 06302d2cb5554805b85916741ea10f0c with delete_type TABLET_DATA_DELETED (replaced by tablet b4ef27777bc047b886a1be0d4622dcf8) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.885154 8550 ts_tablet_manager.cc:1929] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.885272 8550 log.cc:1199] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/bd50e7d4e28541de9799833c53fe7c02
I20251028 02:59:57.885540 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 429b9cd6b775436a865afd64ec5eae4f with delete_type TABLET_DATA_DELETED (replaced by tablet 69c9f986f7004ee6a0b6bba624ae1f8d) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.885581 8550 ts_tablet_manager.cc:1950] T bd50e7d4e28541de9799833c53fe7c02 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.885954 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet bd50e7d4e28541de9799833c53fe7c02 (table default.Table_424dccd707fc4fa1bd9d01e78cab2f92 [id=dbe3cee1ce8a47fab35d8303c2950b1c]) successfully deleted
I20251028 02:59:57.887986 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet fcab55fd49da41f7a95d46d76a8dac8b (table default.Table_11f1aff812d14b0f8b0004ae5438d7e0 [id=86e0c4d129fc42f1998b2b0f2dbc2135]). Expected version 1 got 0
I20251028 02:59:57.888258 8550 tablet_replica.cc:333] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.888499 8550 raft_consensus.cc:2243] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.890584 8550 raft_consensus.cc:2272] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.891105 8551 ts_tablet_manager.cc:1916] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.891420 8550 ts_tablet_manager.cc:1916] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.893899 8551 ts_tablet_manager.cc:1929] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.894039 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet b46554b0df894beb8141126aa7ee7b8e (table default.Table_176b16f7b1fd4a30991b82efd4ddd497 [id=9e5ad11a9ea34a3a8c629f48142b68f0]). Expected version 1 got 0
I20251028 02:59:57.894690 8551 log.cc:1199] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9583046f549d411a9e341560f428381c
I20251028 02:59:57.895035 8551 ts_tablet_manager.cc:1950] T 9583046f549d411a9e341560f428381c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.895380 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 02695b18936d4f928cd55f9b76b90b2e
I20251028 02:59:57.895677 7966 catalog_manager.cc:6719] ReplaceTablet: tablet 02695b18936d4f928cd55f9b76b90b2e deleted and replaced by tablet ebf449a6d241486a892eba79568c3409
I20251028 02:59:57.895859 8551 tablet_replica.cc:333] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.896070 8551 raft_consensus.cc:2243] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.896183 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 02695b18936d4f928cd55f9b76b90b2e with delete_type TABLET_DATA_DELETED (replaced by tablet ebf449a6d241486a892eba79568c3409) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.896358 8551 raft_consensus.cc:2272] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.896400 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9583046f549d411a9e341560f428381c (table default.Table_8b7624342986474fbd0e31c86947f781 [id=530470c553154d76bf0df08ddfbf2fd0]) successfully deleted
I20251028 02:59:57.897118 8551 ts_tablet_manager.cc:1916] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.897325 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 02695b18936d4f928cd55f9b76b90b2e with delete_type TABLET_DATA_DELETED (replaced by tablet ebf449a6d241486a892eba79568c3409) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.898243 8544 tablet_bootstrap.cc:492] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.898304 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 02695b18936d4f928cd55f9b76b90b2e with delete_type TABLET_DATA_DELETED (replaced by tablet ebf449a6d241486a892eba79568c3409) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.898344 8544 ts_tablet_manager.cc:1403] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20251028 02:59:57.899076 8544 raft_consensus.cc:359] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.899189 8544 raft_consensus.cc:385] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.899223 8544 raft_consensus.cc:740] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.899391 8544 consensus_queue.cc:260] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.899552 9401 raft_consensus.cc:493] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.899621 9401 raft_consensus.cc:515] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.899998 9401 leader_election.cc:290] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.900457 8544 ts_tablet_manager.cc:1434] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251028 02:59:57.900660 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 127e6316879e429998799ba17f0b17d5
I20251028 02:59:57.900684 8544 tablet_bootstrap.cc:492] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.901127 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81b521c9f57844bb9873f6b26b5cf661" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.901227 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81b521c9f57844bb9873f6b26b5cf661" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.901384 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 127e6316879e429998799ba17f0b17d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.901764 8544 tablet_bootstrap.cc:654] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.901923 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 127e6316879e429998799ba17f0b17d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:57.901990 8083 leader_election.cc:343] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:57.902119 8083 leader_election.cc:343] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.902164 8083 leader_election.cc:304] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.902283 9401 raft_consensus.cc:2749] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.902518 9401 raft_consensus.cc:493] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.902585 9401 raft_consensus.cc:515] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.902763 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 127e6316879e429998799ba17f0b17d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.902916 9401 leader_election.cc:290] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:57.903745 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "322cbd8253ce45118ad14e8f8f2f1f74" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:57.903752 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "322cbd8253ce45118ad14e8f8f2f1f74" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:57.905467 8551 ts_tablet_manager.cc:1929] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:57.905584 8551 log.cc:1199] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/06302d2cb5554805b85916741ea10f0c
I20251028 02:59:57.905910 8551 ts_tablet_manager.cc:1950] T 06302d2cb5554805b85916741ea10f0c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 02:59:57.906059 8083 leader_election.cc:343] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:57.906211 8083 leader_election.cc:343] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.906261 8083 leader_election.cc:304] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.906383 9401 raft_consensus.cc:2749] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.906811 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 06302d2cb5554805b85916741ea10f0c (table default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d]) successfully deleted
I20251028 02:59:57.907065 8551 tablet_replica.cc:333] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:57.907218 8550 ts_tablet_manager.cc:1929] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.907246 8551 raft_consensus.cc:2243] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.907405 8551 raft_consensus.cc:2272] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.907933 8386 tablet_service.cc:1505] Processing CreateTablet for tablet df4042c347c24691a0aa0c761b6723c7 (DEFAULT_TABLE table=default.Table_479f49bd28d1417a94221c750005fe3f [id=65ef2e3ee834444085ae1c01494bc460]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.908231 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df4042c347c24691a0aa0c761b6723c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.908898 8263 tablet_service.cc:1505] Processing CreateTablet for tablet df4042c347c24691a0aa0c761b6723c7 (DEFAULT_TABLE table=default.Table_479f49bd28d1417a94221c750005fe3f [id=65ef2e3ee834444085ae1c01494bc460]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.909190 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df4042c347c24691a0aa0c761b6723c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.910107 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 3ca1487a8a8644b695471181c1343545 (DEFAULT_TABLE table=default.Table_7a237cac752f41ea9761c1845b56171d [id=3e9d6e4c7db74602a26edf668800f332]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.910322 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca1487a8a8644b695471181c1343545. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.912516 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 3ca1487a8a8644b695471181c1343545 (DEFAULT_TABLE table=default.Table_7a237cac752f41ea9761c1845b56171d [id=3e9d6e4c7db74602a26edf668800f332]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.912739 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca1487a8a8644b695471181c1343545. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.913573 8263 tablet_service.cc:1505] Processing CreateTablet for tablet f79e2add439c4d36af2e84eacbcdbc8f (DEFAULT_TABLE table=default.Table_abcc95f588d04951b016ffd9ea9800c5 [id=84ed833c41e347df99a0592fd9f9f4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.913789 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f79e2add439c4d36af2e84eacbcdbc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.909355 8113 tablet_service.cc:1505] Processing CreateTablet for tablet df4042c347c24691a0aa0c761b6723c7 (DEFAULT_TABLE table=default.Table_479f49bd28d1417a94221c750005fe3f [id=65ef2e3ee834444085ae1c01494bc460]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.915550 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df4042c347c24691a0aa0c761b6723c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.915526 8400 tablet_service.cc:1505] Processing CreateTablet for tablet f79e2add439c4d36af2e84eacbcdbc8f (DEFAULT_TABLE table=default.Table_abcc95f588d04951b016ffd9ea9800c5 [id=84ed833c41e347df99a0592fd9f9f4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.915738 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f79e2add439c4d36af2e84eacbcdbc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.916571 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 110d3531524a460a8e6c2966ef0c367f (DEFAULT_TABLE table=default.Table_196d7bd17aed4ca7909e5ab6e7653130 [id=dda6b076b7744040a4138677c01846c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.916801 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 110d3531524a460a8e6c2966ef0c367f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.917515 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 110d3531524a460a8e6c2966ef0c367f (DEFAULT_TABLE table=default.Table_196d7bd17aed4ca7909e5ab6e7653130 [id=dda6b076b7744040a4138677c01846c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.914906 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 3ca1487a8a8644b695471181c1343545 (DEFAULT_TABLE table=default.Table_7a237cac752f41ea9761c1845b56171d [id=3e9d6e4c7db74602a26edf668800f332]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.917744 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 110d3531524a460a8e6c2966ef0c367f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.917795 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca1487a8a8644b695471181c1343545. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.918843 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 37a128fff52e4c9d9e1cd863fe80ec9c (DEFAULT_TABLE table=default.Table_cd4df8be533e472bafeb04d0aa635832 [id=1b512207483d41adb36d9d76c3b560f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.919099 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a128fff52e4c9d9e1cd863fe80ec9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.919535 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 37a128fff52e4c9d9e1cd863fe80ec9c (DEFAULT_TABLE table=default.Table_cd4df8be533e472bafeb04d0aa635832 [id=1b512207483d41adb36d9d76c3b560f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.915113 8128 tablet_service.cc:1505] Processing CreateTablet for tablet f79e2add439c4d36af2e84eacbcdbc8f (DEFAULT_TABLE table=default.Table_abcc95f588d04951b016ffd9ea9800c5 [id=84ed833c41e347df99a0592fd9f9f4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.919790 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a128fff52e4c9d9e1cd863fe80ec9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.919910 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f79e2add439c4d36af2e84eacbcdbc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.921275 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 5c499b38a9854136bc82a891ef8a87c2 (DEFAULT_TABLE table=default.Table_1af3c83bed1b4fd2bbc9f6a0b0716489 [id=661c1e9f495b423aab4ea7edd01b9270]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.921492 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c499b38a9854136bc82a891ef8a87c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.923194 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 5c499b38a9854136bc82a891ef8a87c2 (DEFAULT_TABLE table=default.Table_1af3c83bed1b4fd2bbc9f6a0b0716489 [id=661c1e9f495b423aab4ea7edd01b9270]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.923484 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c499b38a9854136bc82a891ef8a87c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.923825 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 5c499b38a9854136bc82a891ef8a87c2 (DEFAULT_TABLE table=default.Table_1af3c83bed1b4fd2bbc9f6a0b0716489 [id=661c1e9f495b423aab4ea7edd01b9270]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.924053 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c499b38a9854136bc82a891ef8a87c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.924167 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_11f1aff812d14b0f8b0004ae5438d7e0 [id=86e0c4d129fc42f1998b2b0f2dbc2135] alter complete (version 1)
I20251028 02:59:57.925240 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet e9ad4ae17665483584ce2d638d1639fb
I20251028 02:59:57.925642 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 37a128fff52e4c9d9e1cd863fe80ec9c (DEFAULT_TABLE table=default.Table_cd4df8be533e472bafeb04d0aa635832 [id=1b512207483d41adb36d9d76c3b560f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.926004 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet b7d3062426434c339be4a745decae729
I20251028 02:59:57.926440 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5d5c68c16c3e4d37be7f9a505ab44549
I20251028 02:59:57.926740 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ad4ae17665483584ce2d638d1639fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.926812 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet b7d3062426434c339be4a745decae729 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.926888 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet b7d3062426434c339be4a745decae729 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.927145 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a128fff52e4c9d9e1cd863fe80ec9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.927608 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d5c68c16c3e4d37be7f9a505ab44549 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.928853 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d5c68c16c3e4d37be7f9a505ab44549 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.929458 8550 log.cc:1199] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/874b6660d86b48ebad03ecc54588da28
I20251028 02:59:57.929766 8550 ts_tablet_manager.cc:1950] T 874b6660d86b48ebad03ecc54588da28 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.929771 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 110d3531524a460a8e6c2966ef0c367f (DEFAULT_TABLE table=default.Table_196d7bd17aed4ca7909e5ab6e7653130 [id=dda6b076b7744040a4138677c01846c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.930073 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 110d3531524a460a8e6c2966ef0c367f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.930327 8550 tablet_replica.cc:333] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.930464 8550 raft_consensus.cc:2243] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.930568 8550 raft_consensus.cc:2272] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.933145 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 1a3132020baa4d9a83a76df511fac5c6 (DEFAULT_TABLE table=default.Table_4675220a671f4306b8f46597ab3ca721 [id=5928881222b6413f84d3accfd8e87692]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.933336 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet b7d3062426434c339be4a745decae729 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.933429 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f06af61617e4be9a753692e9ad4978d with delete_type TABLET_DATA_DELETED (replaced by tablet 26a60624333d43f1b701036b7e9dfdb0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.933460 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d5c68c16c3e4d37be7f9a505ab44549 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.933492 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3132020baa4d9a83a76df511fac5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.933811 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 874b6660d86b48ebad03ecc54588da28 (table default.Table_c7e0f5beff234f17b69504efc678924d [id=181b50c2d29a4b17b8189e3abb10a420]) successfully deleted
I20251028 02:59:57.934288 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ad4ae17665483584ce2d638d1639fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.934306 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 1a3132020baa4d9a83a76df511fac5c6 (DEFAULT_TABLE table=default.Table_4675220a671f4306b8f46597ab3ca721 [id=5928881222b6413f84d3accfd8e87692]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.934505 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet a19eeac727e142579e34a301a6e2db02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.934567 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3132020baa4d9a83a76df511fac5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.934628 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 45572222af854b34a5f26a00f8ad9615 with delete_type TABLET_DATA_DELETED (replaced by tablet d60fc654f6af44b290d46235082a4113) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.934707 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 4902bc3398754802a5c571036cd19fa7 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:57.935125 8544 tablet_bootstrap.cc:492] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.935204 8544 ts_tablet_manager.cc:1403] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.002s
I20251028 02:59:57.935814 8544 raft_consensus.cc:359] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.935945 8544 raft_consensus.cc:385] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.935981 8544 raft_consensus.cc:740] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.936131 8544 consensus_queue.cc:260] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.929464 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 1a3132020baa4d9a83a76df511fac5c6 (DEFAULT_TABLE table=default.Table_4675220a671f4306b8f46597ab3ca721 [id=5928881222b6413f84d3accfd8e87692]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:57.937206 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3132020baa4d9a83a76df511fac5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:57.937565 8544 ts_tablet_manager.cc:1434] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 02:59:57.937628 8552 ts_tablet_manager.cc:1929] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.937702 8552 log.cc:1199] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/bd50e7d4e28541de9799833c53fe7c02
I20251028 02:59:57.937732 8544 tablet_bootstrap.cc:492] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.938766 8544 tablet_bootstrap.cc:654] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.939672 9526 tablet.cc:1722] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.940097 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ad4ae17665483584ce2d638d1639fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.940201 8550 ts_tablet_manager.cc:1916] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.943711 8550 ts_tablet_manager.cc:1929] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.943822 9685 raft_consensus.cc:493] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:57.943889 8550 log.cc:1199] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/d59acfb0e12643ebbb406dbda7bef29b
I20251028 02:59:57.943935 9685 raft_consensus.cc:515] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.944299 8550 ts_tablet_manager.cc:1950] T d59acfb0e12643ebbb406dbda7bef29b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.944342 9685 leader_election.cc:290] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.945051 8550 tablet_replica.cc:333] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.945261 8550 raft_consensus.cc:2243] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.945428 8550 raft_consensus.cc:2272] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.945530 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a808098c70b54ad3a0230352af0b743c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:57.945895 8220 leader_election.cc:343] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.946353 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4902bc3398754802a5c571036cd19fa7
I20251028 02:59:57.946480 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a808098c70b54ad3a0230352af0b743c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:57.946602 8145 raft_consensus.cc:2468] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:57.946669 7961 catalog_manager.cc:6719] ReplaceTablet: tablet 4902bc3398754802a5c571036cd19fa7 deleted and replaced by tablet b42f039c30a048e299c39bcf44033125
I20251028 02:59:57.946822 8550 ts_tablet_manager.cc:1916] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.947119 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 4902bc3398754802a5c571036cd19fa7 with delete_type TABLET_DATA_DELETED (replaced by tablet b42f039c30a048e299c39bcf44033125) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:57.947337 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 4902bc3398754802a5c571036cd19fa7 with delete_type TABLET_DATA_DELETED (replaced by tablet b42f039c30a048e299c39bcf44033125) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.947492 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 4902bc3398754802a5c571036cd19fa7 with delete_type TABLET_DATA_DELETED (replaced by tablet b42f039c30a048e299c39bcf44033125) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:57.947626 8218 leader_election.cc:304] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:57.947692 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet d59acfb0e12643ebbb406dbda7bef29b (table default.Table_e4f1369ad6ad4110b5854544d4a266bd [id=ccea14281b2c420a99cede953a8de44d]) successfully deleted
I20251028 02:59:57.947842 8553 raft_consensus.cc:493] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251028 02:59:57.947865 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ffe824a618714776bfab38679c9565e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffe824a618714776bfab38679c9565e0 already in progress: creating tablet
I20251028 02:59:57.947916 8553 raft_consensus.cc:515] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.948264 8553 leader_election.cc:290] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.948385 8553 raft_consensus.cc:2804] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:57.948426 8553 raft_consensus.cc:493] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:57.948459 8553 raft_consensus.cc:3060] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.948758 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0f428e5c6fe43fc8f78761d868ddd09" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:57.949090 8220 leader_election.cc:343] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.949426 8550 ts_tablet_manager.cc:1929] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.949388 8553 raft_consensus.cc:515] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.949486 8550 log.cc:1199] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/f08a6e30211a4d59934c36c7a26ac983
I20251028 02:59:57.949687 8550 ts_tablet_manager.cc:1950] T f08a6e30211a4d59934c36c7a26ac983 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.949822 8553 leader_election.cc:290] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:57.950297 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a808098c70b54ad3a0230352af0b743c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:57.950388 8144 raft_consensus.cc:3060] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:57.950662 8551 ts_tablet_manager.cc:1916] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.951216 8144 raft_consensus.cc:2468] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:57.951507 8218 leader_election.cc:304] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:57.951988 9685 raft_consensus.cc:2804] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:57.952136 9685 raft_consensus.cc:697] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:57.952332 9685 consensus_queue.cc:237] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.948251 8546 tablet_bootstrap.cc:492] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:57.952864 8546 ts_tablet_manager.cc:1403] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.087s user 0.001s sys 0.001s
I20251028 02:59:57.952982 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a808098c70b54ad3a0230352af0b743c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
W20251028 02:59:57.953303 8218 leader_election.cc:343] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: c0f428e5c6fe43fc8f78761d868ddd09
W20251028 02:59:57.953377 8220 leader_election.cc:343] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:57.953383 8218 leader_election.cc:304] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:57.953969 8553 raft_consensus.cc:2749] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:57.953999 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet f08a6e30211a4d59934c36c7a26ac983 (table default.Table_c62970e81d90490ebe817e6ff5c68b97 [id=b9c53ab5608f49309950ed79b8272710]) successfully deleted
I20251028 02:59:57.954216 8550 tablet_replica.cc:333] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.954407 8550 raft_consensus.cc:2243] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.954669 8550 raft_consensus.cc:2272] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.955132 8550 ts_tablet_manager.cc:1916] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.955169 8552 ts_tablet_manager.cc:1950] T bd50e7d4e28541de9799833c53fe7c02 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.957201 9685 consensus_queue.cc:1048] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:57.957471 7961 catalog_manager.cc:5649] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:57.957718 8550 ts_tablet_manager.cc:1929] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.957794 8550 log.cc:1199] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/429b9cd6b775436a865afd64ec5eae4f
I20251028 02:59:57.957993 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet a808098c70b54ad3a0230352af0b743c (table default.Table_9e950f6a07524cb0b5a0d3d967668ad6 [id=5dd4cb94fa384facbdd1aef677a4b498]). Expected version 1 got 0
I20251028 02:59:57.958086 8550 ts_tablet_manager.cc:1950] T 429b9cd6b775436a865afd64ec5eae4f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.958309 8546 raft_consensus.cc:359] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.958449 8546 raft_consensus.cc:385] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.958516 8546 raft_consensus.cc:740] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.958674 8546 consensus_queue.cc:260] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:57.958817 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 429b9cd6b775436a865afd64ec5eae4f (table default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]) successfully deleted
I20251028 02:59:57.959014 8546 ts_tablet_manager.cc:1434] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251028 02:59:57.961019 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]). Expected version 2 got 0
I20251028 02:59:57.963807 8552 tablet_replica.cc:333] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.963954 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet bd50e7d4e28541de9799833c53fe7c02 (table default.Table_424dccd707fc4fa1bd9d01e78cab2f92 [id=dbe3cee1ce8a47fab35d8303c2950b1c]) successfully deleted
I20251028 02:59:57.964020 8552 raft_consensus.cc:2243] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.964231 8552 raft_consensus.cc:2272] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.965022 8552 ts_tablet_manager.cc:1916] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.968828 8552 ts_tablet_manager.cc:1929] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.968988 8552 log.cc:1199] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c0f428e5c6fe43fc8f78761d868ddd09
I20251028 02:59:57.969260 8552 ts_tablet_manager.cc:1950] T c0f428e5c6fe43fc8f78761d868ddd09 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.969480 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_06126295f11f4b82bb6ebec7c701f8ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:57.969923 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06126295f11f4b82bb6ebec7c701f8ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.971422 8550 tablet_replica.cc:333] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.971671 8550 raft_consensus.cc:2243] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.971904 8550 raft_consensus.cc:2272] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.970906 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_83a9f07e5a6c457eab241ff1289b95f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.972555 8550 ts_tablet_manager.cc:1916] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:57.972651 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83a9f07e5a6c457eab241ff1289b95f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.973482 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 81b521c9f57844bb9873f6b26b5cf661 (table default.Table_fd3daefb8df048e7bea5949c30da1e1f [id=5f82b9596e5842deb4d45e19c91629bb]). Expected version 1 got 0
I20251028 02:59:57.971338 8546 tablet_bootstrap.cc:492] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:57.975123 8550 ts_tablet_manager.cc:1929] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:57.975198 8550 log.cc:1199] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/06302d2cb5554805b85916741ea10f0c
I20251028 02:59:57.975782 8550 ts_tablet_manager.cc:1950] T 06302d2cb5554805b85916741ea10f0c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.975785 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c061797d60f14b298c963858268b5e60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:57.975888 8546 tablet_bootstrap.cc:654] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:57.976287 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c061797d60f14b298c963858268b5e60 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:57.976832 8544 tablet_bootstrap.cc:492] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:57.976933 8544 ts_tablet_manager.cc:1403] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.000s
I20251028 02:59:57.977495 8550 tablet_replica.cc:333] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.977445 8544 raft_consensus.cc:359] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.977553 8544 raft_consensus.cc:385] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:57.977607 8544 raft_consensus.cc:740] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:57.977646 8550 raft_consensus.cc:2243] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.977766 8550 raft_consensus.cc:2272] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.977739 8544 consensus_queue.cc:260] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:57.978057 8544 ts_tablet_manager.cc:1434] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:57.978273 8544 tablet_bootstrap.cc:492] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:57.979233 8544 tablet_bootstrap.cc:654] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:57.979290 8550 ts_tablet_manager.cc:1916] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.981523 8550 ts_tablet_manager.cc:1929] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.981606 8550 log.cc:1199] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3f06af61617e4be9a753692e9ad4978d
I20251028 02:59:57.981853 8550 ts_tablet_manager.cc:1950] T 3f06af61617e4be9a753692e9ad4978d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:57.982730 8550 tablet_replica.cc:333] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:57.982889 8550 raft_consensus.cc:2243] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.982987 8550 raft_consensus.cc:2272] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.983497 8550 ts_tablet_manager.cc:1916] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.984807 9551 tablet.cc:1722] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.985802 8552 tablet_replica.cc:333] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.985965 8552 raft_consensus.cc:2243] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:57.986112 8552 pending_rounds.cc:70] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 02:59:57.986140 8552 pending_rounds.cc:77] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597149729386496 op_type: NO_OP noop_request { }
I20251028 02:59:57.986192 8552 raft_consensus.cc:2889] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:57.986240 8552 raft_consensus.cc:2272] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.976423 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 81b521c9f57844bb9873f6b26b5cf661 (table default.Table_fd3daefb8df048e7bea5949c30da1e1f [id=5f82b9596e5842deb4d45e19c91629bb]). Expected version 1 got 0
I20251028 02:59:57.976581 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 06302d2cb5554805b85916741ea10f0c (table default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d]) successfully deleted
I20251028 02:59:57.983230 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5affbd47e83d40b2b4628828251725ac" } new_table_name: "default.Table_d51065c1d57c4857bc5765d1baf9da06" modify_external_catalogs: true
W20251028 02:59:57.988062 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 02695b18936d4f928cd55f9b76b90b2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02695b18936d4f928cd55f9b76b90b2e already in progress: creating tablet
W20251028 02:59:57.988194 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 127e6316879e429998799ba17f0b17d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 127e6316879e429998799ba17f0b17d5 already in progress: creating tablet
W20251028 02:59:57.988291 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet b7d3062426434c339be4a745decae729 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7d3062426434c339be4a745decae729 already in progress: creating tablet
I20251028 02:59:57.987120 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c0f428e5c6fe43fc8f78761d868ddd09 (table default.Table_594bc3babcbf48e4b5d9490f18936563 [id=0ee1306a57fd419497c97f844e58157e]) successfully deleted
I20251028 02:59:57.987509 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5da833b34d9a44cf9f7b9ad381606ec1" } modify_external_catalogs: true
W20251028 02:59:57.988404 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 5d5c68c16c3e4d37be7f9a505ab44549 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d5c68c16c3e4d37be7f9a505ab44549 already in progress: creating tablet
I20251028 02:59:57.988432 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5da833b34d9a44cf9f7b9ad381606ec1" } modify_external_catalogs: true
I20251028 02:59:57.988481 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3f06af61617e4be9a753692e9ad4978d (table default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3]) successfully deleted
W20251028 02:59:57.988608 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e9ad4ae17665483584ce2d638d1639fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9ad4ae17665483584ce2d638d1639fb already in progress: creating tablet
I20251028 02:59:57.990279 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet aa0776d1a8ef43449309221b86ed3923 with delete_type TABLET_DATA_DELETED (replaced by tablet ad926e9ee0044c62a7abbfca959a4f56) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:57.992029 9529 tablet.cc:1722] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:57.995122 8552 ts_tablet_manager.cc:1916] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.997059 8551 ts_tablet_manager.cc:1929] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.997515 8552 ts_tablet_manager.cc:1929] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:57.997609 8552 log.cc:1199] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/874b6660d86b48ebad03ecc54588da28
I20251028 02:59:57.997884 8552 ts_tablet_manager.cc:1950] T 874b6660d86b48ebad03ecc54588da28 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:57.997996 8551 log.cc:1199] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/429b9cd6b775436a865afd64ec5eae4f
I20251028 02:59:57.998371 8551 ts_tablet_manager.cc:1950] T 429b9cd6b775436a865afd64ec5eae4f P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:57.998595 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 874b6660d86b48ebad03ecc54588da28 (table default.Table_c7e0f5beff234f17b69504efc678924d [id=181b50c2d29a4b17b8189e3abb10a420]) successfully deleted
I20251028 02:59:57.998839 8552 tablet_replica.cc:333] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:57.998981 8552 raft_consensus.cc:2243] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:57.999104 8552 raft_consensus.cc:2272] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.999527 8552 ts_tablet_manager.cc:1916] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:57.999545 8550 ts_tablet_manager.cc:1929] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:57.999614 8550 log.cc:1199] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a19eeac727e142579e34a301a6e2db02
I20251028 02:59:57.999855 8550 ts_tablet_manager.cc:1950] T a19eeac727e142579e34a301a6e2db02 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.000449 8550 tablet_replica.cc:333] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.000581 8550 raft_consensus.cc:2243] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.000682 8550 raft_consensus.cc:2272] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:57.998692 9526 tablet.cc:1722] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:58.002565 8545 tablet_bootstrap.cc:492] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.002646 8545 ts_tablet_manager.cc:1403] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.146s user 0.003s sys 0.000s
I20251028 02:59:58.002866 8552 ts_tablet_manager.cc:1929] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:58.002929 8552 log.cc:1199] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/429b9cd6b775436a865afd64ec5eae4f
I20251028 02:59:58.002897 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a19eeac727e142579e34a301a6e2db02 (table default.Table_0ac514a2be6a4e1b904149fd8fdc8eb3 [id=effb15303ea6421a8eebaa32a124678e]) successfully deleted
I20251028 02:59:58.003181 8552 ts_tablet_manager.cc:1950] T 429b9cd6b775436a865afd64ec5eae4f P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.003165 8545 raft_consensus.cc:359] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.003257 8545 raft_consensus.cc:385] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.003285 8545 raft_consensus.cc:740] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.003427 8545 consensus_queue.cc:260] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.003860 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 429b9cd6b775436a865afd64ec5eae4f (table default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]) successfully deleted
I20251028 02:59:58.004087 8545 ts_tablet_manager.cc:1434] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:58.005034 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_47e366fafa11436db14f8076fd1f1ba2" } new_table_name: "default.Table_7ceb18d43d6c41d2b7809c8c97a5f2e7" modify_external_catalogs: true
I20251028 02:59:58.005298 8552 tablet_replica.cc:333] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.005887 8544 tablet_bootstrap.cc:492] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.005987 8544 ts_tablet_manager.cc:1403] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20251028 02:59:58.006412 8550 ts_tablet_manager.cc:1916] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.006646 8544 raft_consensus.cc:359] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.006754 8544 raft_consensus.cc:385] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.006786 8544 raft_consensus.cc:740] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.006904 8544 consensus_queue.cc:260] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.007165 8544 ts_tablet_manager.cc:1434] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.007385 8544 tablet_bootstrap.cc:492] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.007413 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b58c610c254941d2837f6594f592865e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.007732 8545 tablet_bootstrap.cc:492] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:58.007859 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b58c610c254941d2837f6594f592865e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.008407 8544 tablet_bootstrap.cc:654] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.008638 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_514bd1c4b2974dc28f976754be3f2c1c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.008955 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_514bd1c4b2974dc28f976754be3f2c1c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.009560 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_60724e0676e54a429b5cd8d525030e29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.009899 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60724e0676e54a429b5cd8d525030e29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.005450 8552 raft_consensus.cc:2243] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.010465 8552 raft_consensus.cc:2272] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.011183 8550 ts_tablet_manager.cc:1929] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.011253 8550 log.cc:1199] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/45572222af854b34a5f26a00f8ad9615
I20251028 02:59:58.011511 8550 ts_tablet_manager.cc:1950] T 45572222af854b34a5f26a00f8ad9615 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.012831 8552 ts_tablet_manager.cc:1916] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.013141 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 45572222af854b34a5f26a00f8ad9615 (table default.Table_ba93ba86d1664f40b4e919124f42b551 [id=94895dadf0e34a19a454e8755c61c785]) successfully deleted
W20251028 02:59:58.014246 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4902bc3398754802a5c571036cd19fa7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4902bc3398754802a5c571036cd19fa7 already in progress: creating tablet
I20251028 02:59:58.014500 8545 tablet_bootstrap.cc:654] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.016193 8552 ts_tablet_manager.cc:1929] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:58.016279 8552 log.cc:1199] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/06302d2cb5554805b85916741ea10f0c
I20251028 02:59:58.016527 8552 ts_tablet_manager.cc:1950] T 06302d2cb5554805b85916741ea10f0c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.016986 8550 tablet_replica.cc:333] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.017150 8550 raft_consensus.cc:2243] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.017166 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 06302d2cb5554805b85916741ea10f0c (table default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d]) successfully deleted
I20251028 02:59:58.017273 8550 raft_consensus.cc:2272] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.018390 8544 tablet_bootstrap.cc:492] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.018479 8544 ts_tablet_manager.cc:1403] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:58.018931 8544 raft_consensus.cc:359] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.019044 8544 raft_consensus.cc:385] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.019079 8544 raft_consensus.cc:740] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.019210 8544 consensus_queue.cc:260] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.019500 8544 ts_tablet_manager.cc:1434] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.019691 8544 tablet_bootstrap.cc:492] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.020720 8544 tablet_bootstrap.cc:654] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.023229 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet a808098c70b54ad3a0230352af0b743c (table default.Table_9e950f6a07524cb0b5a0d3d967668ad6 [id=5dd4cb94fa384facbdd1aef677a4b498]). Expected version 1 got 0
I20251028 02:59:58.023701 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 429b9cd6b775436a865afd64ec5eae4f (table default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]) successfully deleted
W20251028 02:59:58.023823 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 02695b18936d4f928cd55f9b76b90b2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02695b18936d4f928cd55f9b76b90b2e already in progress: creating tablet
W20251028 02:59:58.023902 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 127e6316879e429998799ba17f0b17d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 127e6316879e429998799ba17f0b17d5 already in progress: creating tablet
I20251028 02:59:58.023907 8544 tablet_bootstrap.cc:492] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
W20251028 02:59:58.023979 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet b7d3062426434c339be4a745decae729 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7d3062426434c339be4a745decae729 already in progress: creating tablet
I20251028 02:59:58.023994 8544 ts_tablet_manager.cc:1403] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
W20251028 02:59:58.024052 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 5d5c68c16c3e4d37be7f9a505ab44549 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d5c68c16c3e4d37be7f9a505ab44549 already in progress: creating tablet
W20251028 02:59:58.024132 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet e9ad4ae17665483584ce2d638d1639fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9ad4ae17665483584ce2d638d1639fb already in progress: creating tablet
W20251028 02:59:58.024222 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4902bc3398754802a5c571036cd19fa7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4902bc3398754802a5c571036cd19fa7 already in progress: creating tablet
I20251028 02:59:58.024799 8550 ts_tablet_manager.cc:1916] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.025528 8544 raft_consensus.cc:359] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.025652 8544 raft_consensus.cc:385] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.025692 8544 raft_consensus.cc:740] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.025849 8544 consensus_queue.cc:260] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.026172 8544 ts_tablet_manager.cc:1434] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251028 02:59:58.026461 8544 tablet_bootstrap.cc:492] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.027885 8550 ts_tablet_manager.cc:1929] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.027958 8550 log.cc:1199] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/aa0776d1a8ef43449309221b86ed3923
I20251028 02:59:58.028169 8550 ts_tablet_manager.cc:1950] T aa0776d1a8ef43449309221b86ed3923 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 02:59:58.029000 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 02695b18936d4f928cd55f9b76b90b2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02695b18936d4f928cd55f9b76b90b2e already in progress: creating tablet
W20251028 02:59:58.029450 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 127e6316879e429998799ba17f0b17d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 127e6316879e429998799ba17f0b17d5 already in progress: creating tablet
I20251028 02:59:58.029706 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2bd4558fc315491687a12efce7ee8ab2
W20251028 02:59:58.029834 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet e9ad4ae17665483584ce2d638d1639fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9ad4ae17665483584ce2d638d1639fb already in progress: creating tablet
W20251028 02:59:58.030408 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aa36c6870c1b4a459f0a41c84855a76e consensus state has no leader
I20251028 02:59:58.030526 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 2bd4558fc315491687a12efce7ee8ab2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.030880 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 2bd4558fc315491687a12efce7ee8ab2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.031458 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 2bd4558fc315491687a12efce7ee8ab2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.031466 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet aa0776d1a8ef43449309221b86ed3923 (table default.Table_6ad31a79d9f043628bdac1dd2d4c263c [id=4570d7ddf33c447fb40d6411ea47e52a]) successfully deleted
I20251028 02:59:58.031876 8544 tablet_bootstrap.cc:654] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.032513 8551 tablet_replica.cc:333] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.032661 8551 raft_consensus.cc:2243] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:58.032886 8551 raft_consensus.cc:2272] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:58.033416 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet b7d3062426434c339be4a745decae729 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7d3062426434c339be4a745decae729 already in progress: creating tablet
W20251028 02:59:58.033804 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 5d5c68c16c3e4d37be7f9a505ab44549 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d5c68c16c3e4d37be7f9a505ab44549 already in progress: creating tablet
I20251028 02:59:58.033914 8551 ts_tablet_manager.cc:1916] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:58.034173 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4902bc3398754802a5c571036cd19fa7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4902bc3398754802a5c571036cd19fa7 already in progress: creating tablet
I20251028 02:59:58.034361 8552 tablet_replica.cc:333] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.034495 8552 raft_consensus.cc:2243] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.034629 8552 raft_consensus.cc:2272] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.034807 8550 tablet_replica.cc:333] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.034940 8550 raft_consensus.cc:2243] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.035125 8550 raft_consensus.cc:2272] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.035725 8552 ts_tablet_manager.cc:1916] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.036309 8550 ts_tablet_manager.cc:1916] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.035840 8553 raft_consensus.cc:493] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.037153 8553 raft_consensus.cc:515] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.038131 8553 leader_election.cc:290] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 02:59:58.038565 8220 leader_election.cc:343] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: ab1ec08da9f64109a3fe9400fe4e366c
I20251028 02:59:58.038621 8552 ts_tablet_manager.cc:1929] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:58.038681 8552 log.cc:1199] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/2bd4558fc315491687a12efce7ee8ab2
I20251028 02:59:58.038918 8552 ts_tablet_manager.cc:1950] T 2bd4558fc315491687a12efce7ee8ab2 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 02:59:58.039693 8218 leader_election.cc:343] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: ab1ec08da9f64109a3fe9400fe4e366c
I20251028 02:59:58.039759 8218 leader_election.cc:304] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:58.040191 9685 raft_consensus.cc:2749] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251028 02:59:58.040946 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 2bd4558fc315491687a12efce7ee8ab2 (table default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9]) successfully deleted
I20251028 02:59:58.043047 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_aee4903456c34845a3085d4f181b13ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.043358 8550 ts_tablet_manager.cc:1929] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:58.043437 8550 log.cc:1199] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2bd4558fc315491687a12efce7ee8ab2
W20251028 02:59:58.043494 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aee4903456c34845a3085d4f181b13ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.043700 8550 ts_tablet_manager.cc:1950] T 2bd4558fc315491687a12efce7ee8ab2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.044164 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2bd4558fc315491687a12efce7ee8ab2 (table default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9]) successfully deleted
I20251028 02:59:58.044410 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0ac5b50f10cd4c52b002d3906b21b3a7" } new_table_name: "default.Table_85a3d44a226b47a4b3125fe86737a4b0" modify_external_catalogs: true
I20251028 02:59:58.045346 8551 ts_tablet_manager.cc:1929] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:58.045436 8551 log.cc:1199] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2bd4558fc315491687a12efce7ee8ab2
I20251028 02:59:58.045699 8551 ts_tablet_manager.cc:1950] T 2bd4558fc315491687a12efce7ee8ab2 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.046674 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2bd4558fc315491687a12efce7ee8ab2 (table default.Table_5da833b34d9a44cf9f7b9ad381606ec1 [id=a8812c2764af480bbc45bf74644d4fb9]) successfully deleted
I20251028 02:59:58.048285 9685 consensus_queue.cc:1048] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:58.050092 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff] alter complete (version 1)
I20251028 02:59:58.052847 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4ebe7022d46e41b68350a3b22f24a919"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.053249 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ebe7022d46e41b68350a3b22f24a919 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.054035 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_03e0732cf61746cb89213075b54e35cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.054365 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03e0732cf61746cb89213075b54e35cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:58.056237 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 12eecf12cac641eaad5a9b5af5890941 consensus state has no leader
I20251028 02:59:58.056479 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_9090533beeb54c4cb4a11365200d3b4e" } new_table_name: "default.Table_059639c7191541caa3dbb30b333d579f" modify_external_catalogs: true
I20251028 02:59:58.056695 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ba93ba86d1664f40b4e919124f42b551" } new_table_name: "default.Table_a387569232e54415a0dd77e906ddcee4" modify_external_catalogs: true
I20251028 02:59:58.058429 9551 tablet.cc:1722] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:58.058657 9529 tablet.cc:1722] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:58.068626 8562 raft_consensus.cc:493] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.068751 8562 raft_consensus.cc:515] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.069171 8562 leader_election.cc:290] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:58.069711 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93ef23900c0e4498934208352eb706f8" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.070037 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93ef23900c0e4498934208352eb706f8" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:58.070127 8083 leader_election.cc:343] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:58.070425 8083 leader_election.cc:343] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.070523 8083 leader_election.cc:304] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.070667 8562 raft_consensus.cc:2749] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.075459 8545 tablet_bootstrap.cc:492] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.075578 8545 ts_tablet_manager.cc:1403] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.071s user 0.001s sys 0.002s
I20251028 02:59:58.076061 8545 raft_consensus.cc:359] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.076184 8545 raft_consensus.cc:385] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.076224 8545 raft_consensus.cc:740] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.076342 8545 consensus_queue.cc:260] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.076633 8545 ts_tablet_manager.cc:1434] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.076814 8545 tablet_bootstrap.cc:492] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.077757 8545 tablet_bootstrap.cc:654] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.078847 9733 tablet.cc:1722] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:58.083662 8545 tablet_bootstrap.cc:492] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.083771 8545 ts_tablet_manager.cc:1403] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20251028 02:59:58.084339 8545 raft_consensus.cc:359] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.084450 8545 raft_consensus.cc:385] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.084484 8545 raft_consensus.cc:740] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.085063 8545 consensus_queue.cc:260] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.085529 8545 ts_tablet_manager.cc:1434] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 02:59:58.086117 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet d39008ac6324451a9d1ed54a2ccc3abf from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.089648 7963 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet c21f794ec1284f5d99521954ed7ba9c9 (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]). Expected version 1 got 0
I20251028 02:59:58.089823 7963 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]). Expected version 2 got 0
I20251028 02:59:58.090309 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]). Expected version 2 got 0
I20251028 02:59:58.092515 8396 tablet_service.cc:1505] Processing CreateTablet for tablet ebf449a6d241486a892eba79568c3409 (DEFAULT_TABLE table=default.Table_e2683e2b014947bd87e048f63aa666f1 [id=058a75d461a0497a9268f8edf3506ece]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.092686 8398 tablet_service.cc:1505] Processing CreateTablet for tablet b42f039c30a048e299c39bcf44033125 (DEFAULT_TABLE table=default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.092710 8386 tablet_service.cc:1505] Processing CreateTablet for tablet fbf0114b95a4492380d4048d8f4c97e4 (DEFAULT_TABLE table=default.Table_c175a83383134da183a6ca9e861ea2ba [id=10af3345ea1f43668a7e444f6aa5ebaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.092813 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebf449a6d241486a892eba79568c3409. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.092962 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 69c9f986f7004ee6a0b6bba624ae1f8d (DEFAULT_TABLE table=default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.093021 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b42f039c30a048e299c39bcf44033125. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.093182 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 3872cb93268a4e86b571339ce2d7b127 (DEFAULT_TABLE table=default.Table_4673cd2bd8f2439c94aac247f1ce8f12 [id=c982ff397653439c8b3153144b7f2f3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.093416 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 6233be1842804618a5efc76a75cd85f5 (DEFAULT_TABLE table=default.Table_a19abcac1bc24d7aa75a6032b0689b1b [id=90acec86b67e49b4ae6ff156698bed74]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.093616 8384 tablet_service.cc:1505] Processing CreateTablet for tablet b4ef27777bc047b886a1be0d4622dcf8 (DEFAULT_TABLE table=default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.093775 8259 tablet_service.cc:1505] Processing CreateTablet for tablet ebf449a6d241486a892eba79568c3409 (DEFAULT_TABLE table=default.Table_e2683e2b014947bd87e048f63aa666f1 [id=058a75d461a0497a9268f8edf3506ece]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.093873 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 6233be1842804618a5efc76a75cd85f5 (DEFAULT_TABLE table=default.Table_a19abcac1bc24d7aa75a6032b0689b1b [id=90acec86b67e49b4ae6ff156698bed74]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.094038 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebf449a6d241486a892eba79568c3409. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.094267 8256 tablet_service.cc:1505] Processing CreateTablet for tablet b42f039c30a048e299c39bcf44033125 (DEFAULT_TABLE table=default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.094491 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b42f039c30a048e299c39bcf44033125. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.094836 8545 tablet_bootstrap.cc:492] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.095407 8261 tablet_service.cc:1505] Processing CreateTablet for tablet fbf0114b95a4492380d4048d8f4c97e4 (DEFAULT_TABLE table=default.Table_c175a83383134da183a6ca9e861ea2ba [id=10af3345ea1f43668a7e444f6aa5ebaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.095623 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbf0114b95a4492380d4048d8f4c97e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.096071 8545 tablet_bootstrap.cc:654] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.096460 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 3872cb93268a4e86b571339ce2d7b127 (DEFAULT_TABLE table=default.Table_4673cd2bd8f2439c94aac247f1ce8f12 [id=c982ff397653439c8b3153144b7f2f3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.096575 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6233be1842804618a5efc76a75cd85f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.096823 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c9f986f7004ee6a0b6bba624ae1f8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.097692 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 69c9f986f7004ee6a0b6bba624ae1f8d (DEFAULT_TABLE table=default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.097898 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c9f986f7004ee6a0b6bba624ae1f8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.099481 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3872cb93268a4e86b571339ce2d7b127. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.099524 8249 tablet_service.cc:1505] Processing CreateTablet for tablet b4ef27777bc047b886a1be0d4622dcf8 (DEFAULT_TABLE table=default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.099737 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ef27777bc047b886a1be0d4622dcf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.100189 8544 tablet_bootstrap.cc:492] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.100286 8544 ts_tablet_manager.cc:1403] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.074s user 0.002s sys 0.000s
I20251028 02:59:58.100620 8546 tablet_bootstrap.cc:492] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.100708 8546 ts_tablet_manager.cc:1403] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.129s user 0.002s sys 0.000s
I20251028 02:59:58.101158 8544 raft_consensus.cc:359] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.101281 8544 raft_consensus.cc:385] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.101223 8546 raft_consensus.cc:359] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.101316 8544 raft_consensus.cc:740] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.101330 8546 raft_consensus.cc:385] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.101363 8546 raft_consensus.cc:740] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.101455 8544 consensus_queue.cc:260] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.101498 8546 consensus_queue.cc:260] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.101784 8546 ts_tablet_manager.cc:1434] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.101785 9401 raft_consensus.cc:493] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.101796 8544 ts_tablet_manager.cc:1434] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.101884 9401 raft_consensus.cc:515] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.101974 8546 tablet_bootstrap.cc:492] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.102106 8544 tablet_bootstrap.cc:492] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.102276 9401 leader_election.cc:290] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.103168 8546 tablet_bootstrap.cc:654] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.103169 8544 tablet_bootstrap.cc:654] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.103914 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c21f794ec1284f5d99521954ed7ba9c9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.104180 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ef27777bc047b886a1be0d4622dcf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.104661 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c21f794ec1284f5d99521954ed7ba9c9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:58.106349 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6233be1842804618a5efc76a75cd85f5. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:58.107471 8083 leader_election.cc:343] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:58.107625 8083 leader_election.cc:343] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.107672 8083 leader_election.cc:304] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.107899 8562 raft_consensus.cc:2749] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.108470 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3872cb93268a4e86b571339ce2d7b127. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.110181 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbf0114b95a4492380d4048d8f4c97e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.117583 8546 tablet_bootstrap.cc:492] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.117708 8546 ts_tablet_manager.cc:1403] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 02:59:58.118222 7960 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7] alter complete (version 2)
I20251028 02:59:58.118271 8546 raft_consensus.cc:359] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.118381 8546 raft_consensus.cc:385] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.118414 8546 raft_consensus.cc:740] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.118561 8546 consensus_queue.cc:260] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.118561 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7b7c98fa38d3422489f2813f75fa7deb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.118873 8546 ts_tablet_manager.cc:1434] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 02:59:58.118966 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b7c98fa38d3422489f2813f75fa7deb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.119205 8546 tablet_bootstrap.cc:492] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.119545 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_eca57c87ce0240a0844cc210e4dbdd85" } modify_external_catalogs: true
W20251028 02:59:58.120242 7963 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet c21f794ec1284f5d99521954ed7ba9c9 (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet c21f794ec1284f5d99521954ed7ba9c9 consensus state has no leader
I20251028 02:59:58.121258 8546 tablet_bootstrap.cc:654] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.122608 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 529b865389f94471a6849d17ff206e4c with delete_type TABLET_DATA_DELETED (replaced by tablet 240b7da6f46c4c5ab8087ab1dd6275c5) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.123382 8551 tablet_replica.cc:333] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.126171 8111 tablet_service.cc:1505] Processing CreateTablet for tablet ebf449a6d241486a892eba79568c3409 (DEFAULT_TABLE table=default.Table_e2683e2b014947bd87e048f63aa666f1 [id=058a75d461a0497a9268f8edf3506ece]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.126550 8116 tablet_service.cc:1505] Processing CreateTablet for tablet b4ef27777bc047b886a1be0d4622dcf8 (DEFAULT_TABLE table=default.Table_2af68266ea4d4f288ecc63e064bb69db [id=ef18ab5f49634ad3b7174f0083b4b71d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.126680 8127 tablet_service.cc:1505] Processing CreateTablet for tablet fbf0114b95a4492380d4048d8f4c97e4 (DEFAULT_TABLE table=default.Table_c175a83383134da183a6ca9e861ea2ba [id=10af3345ea1f43668a7e444f6aa5ebaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.126786 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 69c9f986f7004ee6a0b6bba624ae1f8d (DEFAULT_TABLE table=default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.126921 8125 tablet_service.cc:1505] Processing CreateTablet for tablet b42f039c30a048e299c39bcf44033125 (DEFAULT_TABLE table=default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.127135 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 6233be1842804618a5efc76a75cd85f5 (DEFAULT_TABLE table=default.Table_a19abcac1bc24d7aa75a6032b0689b1b [id=90acec86b67e49b4ae6ff156698bed74]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.127156 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b43da9580244346ae512a7bc9b506d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.126550 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 3872cb93268a4e86b571339ce2d7b127 (DEFAULT_TABLE table=default.Table_4673cd2bd8f2439c94aac247f1ce8f12 [id=c982ff397653439c8b3153144b7f2f3b]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:58.128381 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d60fc654f6af44b290d46235082a4113 consensus state has no leader
I20251028 02:59:58.128597 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 3f04c867f0ce4ab99f980cca063abc30 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.130270 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 900973d832c24b62a4554ae9a219cfd8 (table default.Table_f0dafbe2e0a84541a79cf6c887d59619 [id=c4aadbebe43f451ea22467feaa6dafbb]). Expected version 1 got 0
I20251028 02:59:58.130746 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebf449a6d241486a892eba79568c3409. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.130985 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6233be1842804618a5efc76a75cd85f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.131155 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ef27777bc047b886a1be0d4622dcf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.131315 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69c9f986f7004ee6a0b6bba624ae1f8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.131603 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3872cb93268a4e86b571339ce2d7b127. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.132602 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbf0114b95a4492380d4048d8f4c97e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.133904 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b42f039c30a048e299c39bcf44033125. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.135099 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.137429 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6ebfbcbecf7c4975ac638a06859f32d2" } modify_external_catalogs: true
I20251028 02:59:58.137562 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_033167c60f644376b85c64d13503579f" } modify_external_catalogs: true
I20251028 02:59:58.137562 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6ebfbcbecf7c4975ac638a06859f32d2" } modify_external_catalogs: true
I20251028 02:59:58.137703 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_033167c60f644376b85c64d13503579f" } modify_external_catalogs: true
I20251028 02:59:58.138195 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_327424a7e56243ae97d58c0f6fd560fd" } modify_external_catalogs: true
I20251028 02:59:58.138275 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_327424a7e56243ae97d58c0f6fd560fd" } modify_external_catalogs: true
I20251028 02:59:58.138500 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_eca57c87ce0240a0844cc210e4dbdd85" } modify_external_catalogs: true
W20251028 02:59:58.138799 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4eaa472029a94052abb49241b7622d0f consensus state has no leader
I20251028 02:59:58.140125 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet f08a6e30211a4d59934c36c7a26ac983 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.140250 8544 tablet_bootstrap.cc:492] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.140337 8544 ts_tablet_manager.cc:1403] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20251028 02:59:58.140798 8544 raft_consensus.cc:359] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.140900 8544 raft_consensus.cc:385] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.140929 8544 raft_consensus.cc:740] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.141050 8544 consensus_queue.cc:260] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.141352 8544 ts_tablet_manager.cc:1434] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:58.125306 8551 raft_consensus.cc:2243] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.141515 8551 raft_consensus.cc:2272] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.141778 8544 tablet_bootstrap.cc:492] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.142845 8544 tablet_bootstrap.cc:654] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.143328 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfc5240893546d7a6a5397cd08044dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.143530 8545 tablet_bootstrap.cc:492] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.143608 8545 ts_tablet_manager.cc:1403] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.049s user 0.000s sys 0.002s
I20251028 02:59:58.143630 8551 ts_tablet_manager.cc:1916] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.144095 8545 raft_consensus.cc:359] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.144188 8545 raft_consensus.cc:385] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.144215 8545 raft_consensus.cc:740] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.144809 8545 consensus_queue.cc:260] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.145148 8545 ts_tablet_manager.cc:1434] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:58.145318 8545 tablet_bootstrap.cc:492] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.146256 8545 tablet_bootstrap.cc:654] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.148756 8553 raft_consensus.cc:493] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.148865 8553 raft_consensus.cc:515] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.149289 8553 leader_election.cc:290] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.150164 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d39008ac6324451a9d1ed54a2ccc3abf
I20251028 02:59:58.150207 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81b521c9f57844bb9873f6b26b5cf661" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:58.150197 9551 tablet.cc:1722] T e7165de68f6c4987a41cd33b5c6298c4 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:58.150449 7961 catalog_manager.cc:6719] ReplaceTablet: tablet d39008ac6324451a9d1ed54a2ccc3abf deleted and replaced by tablet 985a0daa0a544216bbb06b1bc9d19231
W20251028 02:59:58.152225 8220 leader_election.cc:343] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.155365 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81b521c9f57844bb9873f6b26b5cf661" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:58.155519 8144 raft_consensus.cc:2468] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:58.155530 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ee1d686a0e4c47f6b214224047a55747
I20251028 02:59:58.161677 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet ee1d686a0e4c47f6b214224047a55747 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.162818 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet d39008ac6324451a9d1ed54a2ccc3abf with delete_type TABLET_DATA_DELETED (replaced by tablet 985a0daa0a544216bbb06b1bc9d19231) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.163357 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet fd8c04efd8c2456b9c7643641c7fbc0d
I20251028 02:59:58.163901 8218 leader_election.cc:304] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.164175 9685 raft_consensus.cc:2804] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:58.164239 9685 raft_consensus.cc:493] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:58.164280 9685 raft_consensus.cc:3060] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.164615 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet d39008ac6324451a9d1ed54a2ccc3abf with delete_type TABLET_DATA_DELETED (replaced by tablet 985a0daa0a544216bbb06b1bc9d19231) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.164696 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet fd8c04efd8c2456b9c7643641c7fbc0d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.164744 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 83059074c365440b84e39cf7a5553eb2 with delete_type TABLET_DATA_DELETED (replaced by tablet c120a3d56f6b41c28c7f7dcd84359a60) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.165551 9685 raft_consensus.cc:515] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.165731 7964 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_85a3d44a226b47a4b3125fe86737a4b0 [id=bfbe8c47585048d2a8d3d5f3d62e0c21] alter complete (version 1)
I20251028 02:59:58.165975 9685 leader_election.cc:290] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.165993 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet fd8c04efd8c2456b9c7643641c7fbc0d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.166117 8249 tablet_service.cc:1552] Processing DeleteTablet for tablet ee1d686a0e4c47f6b214224047a55747 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.166324 8552 tablet_replica.cc:333] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.166457 8552 raft_consensus.cc:2243] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.166548 8249 tablet_service.cc:1552] Processing DeleteTablet for tablet fd8c04efd8c2456b9c7643641c7fbc0d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.166603 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 8720ea8c68904d57a3fc5c0e94d338ca from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.166600 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81b521c9f57844bb9873f6b26b5cf661" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:58.166656 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet d39008ac6324451a9d1ed54a2ccc3abf with delete_type TABLET_DATA_DELETED (replaced by tablet 985a0daa0a544216bbb06b1bc9d19231) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.166803 8550 tablet_replica.cc:333] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.166851 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 02f1a2d489d74e33ab991611fd55966d from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.166942 8550 raft_consensus.cc:2243] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:58.167014 8220 leader_election.cc:343] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.167037 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet ee1d686a0e4c47f6b214224047a55747 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.167059 8550 raft_consensus.cc:2272] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.167191 8552 raft_consensus.cc:2272] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.167748 8552 ts_tablet_manager.cc:1916] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.167968 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dfc5240893546d7a6a5397cd08044dd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.169090 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81b521c9f57844bb9873f6b26b5cf661" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:58.169198 8144 raft_consensus.cc:3060] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.170332 8144 raft_consensus.cc:2468] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:58.171415 8218 leader_election.cc:304] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.171671 9685 raft_consensus.cc:2804] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:58.171754 9685 raft_consensus.cc:697] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:58.171900 9685 consensus_queue.cc:237] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.173241 8386 tablet_service.cc:1505] Processing CreateTablet for tablet cd33baf1eaa84510bd7a81cfd879b7b9 (DEFAULT_TABLE table=default.Table_b58c610c254941d2837f6594f592865e [id=c2874698a8334d71a28c1cfe35f43888]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.173442 8551 ts_tablet_manager.cc:1929] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.173514 8551 log.cc:1199] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/529b865389f94471a6849d17ff206e4c
I20251028 02:59:58.173539 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd33baf1eaa84510bd7a81cfd879b7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.173728 8551 ts_tablet_manager.cc:1950] T 529b865389f94471a6849d17ff206e4c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.176259 8116 tablet_service.cc:1505] Processing CreateTablet for tablet cd33baf1eaa84510bd7a81cfd879b7b9 (DEFAULT_TABLE table=default.Table_b58c610c254941d2837f6594f592865e [id=c2874698a8334d71a28c1cfe35f43888]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.176556 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd33baf1eaa84510bd7a81cfd879b7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.178598 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 320140b412dc4c5fb87229cc81d6ea2a (DEFAULT_TABLE table=default.Table_83a9f07e5a6c457eab241ff1289b95f6 [id=67714d889962431bbf16786533b6840f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.178772 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 99cc3e2b1eef4137bac27c629600f43b (DEFAULT_TABLE table=default.Table_514bd1c4b2974dc28f976754be3f2c1c [id=f8d93791c70842e79584bb358f898287]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.178846 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 320140b412dc4c5fb87229cc81d6ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.178891 8113 tablet_service.cc:1505] Processing CreateTablet for tablet e9857ba66e9e4938abe00b1619a275af (DEFAULT_TABLE table=default.Table_60724e0676e54a429b5cd8d525030e29 [id=3f7bfc9e9f3543f09675bbb31ad2079c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.179018 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99cc3e2b1eef4137bac27c629600f43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.179095 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 391608f115e845e88d07ae08ea543e8e (DEFAULT_TABLE table=default.Table_c061797d60f14b298c963858268b5e60 [id=b5dc2cf2c06e4088a2ce80219d6d675f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.179319 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9857ba66e9e4938abe00b1619a275af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.179431 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 391608f115e845e88d07ae08ea543e8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.180116 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 415f127c97114981b32118835d81aa45 (DEFAULT_TABLE table=default.Table_06126295f11f4b82bb6ebec7c701f8ae [id=5e7922fc04a94f05bbdcaf7a4e1437d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.181001 8550 ts_tablet_manager.cc:1916] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.181365 8552 ts_tablet_manager.cc:1929] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.181442 8552 log.cc:1199] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/f08a6e30211a4d59934c36c7a26ac983
I20251028 02:59:58.181658 8552 ts_tablet_manager.cc:1950] T f08a6e30211a4d59934c36c7a26ac983 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.183151 8550 ts_tablet_manager.cc:1929] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.183220 8550 log.cc:1199] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3b43da9580244346ae512a7bc9b506d3
I20251028 02:59:58.183737 8260 tablet_service.cc:1505] Processing CreateTablet for tablet cd33baf1eaa84510bd7a81cfd879b7b9 (DEFAULT_TABLE table=default.Table_b58c610c254941d2837f6594f592865e [id=c2874698a8334d71a28c1cfe35f43888]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.184017 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd33baf1eaa84510bd7a81cfd879b7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.185031 8553 raft_consensus.cc:493] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.185108 8553 raft_consensus.cc:515] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.185519 8553 leader_election.cc:290] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.186024 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b46554b0df894beb8141126aa7ee7b8e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:58.186410 8220 leader_election.cc:343] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.186790 8552 tablet_replica.cc:333] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.186929 8552 raft_consensus.cc:2243] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.187049 8552 raft_consensus.cc:2272] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.187039 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b46554b0df894beb8141126aa7ee7b8e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:58.187161 8144 raft_consensus.cc:2468] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:58.186939 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 320140b412dc4c5fb87229cc81d6ea2a (DEFAULT_TABLE table=default.Table_83a9f07e5a6c457eab241ff1289b95f6 [id=67714d889962431bbf16786533b6840f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.187486 8218 leader_election.cc:304] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.187619 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 320140b412dc4c5fb87229cc81d6ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.187685 8553 raft_consensus.cc:2804] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:58.187731 8553 raft_consensus.cc:493] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:58.187767 8553 raft_consensus.cc:3060] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.188761 8553 raft_consensus.cc:515] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.189136 8553 leader_election.cc:290] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.189572 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b46554b0df894beb8141126aa7ee7b8e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:58.189889 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 320140b412dc4c5fb87229cc81d6ea2a (DEFAULT_TABLE table=default.Table_83a9f07e5a6c457eab241ff1289b95f6 [id=67714d889962431bbf16786533b6840f]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:58.189960 8220 leader_election.cc:343] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.190163 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 320140b412dc4c5fb87229cc81d6ea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.190398 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b46554b0df894beb8141126aa7ee7b8e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:58.190511 8144 raft_consensus.cc:3060] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.190656 8552 ts_tablet_manager.cc:1916] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.192109 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 415f127c97114981b32118835d81aa45. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.192153 8144 raft_consensus.cc:2468] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:58.192160 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9
I20251028 02:59:58.192428 8218 leader_election.cc:304] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.192625 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8720ea8c68904d57a3fc5c0e94d338ca
I20251028 02:59:58.192636 8553 raft_consensus.cc:2804] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:58.192708 8553 raft_consensus.cc:697] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:58.192875 7967 catalog_manager.cc:6719] ReplaceTablet: tablet 8720ea8c68904d57a3fc5c0e94d338ca deleted and replaced by tablet 30232298155542ecb41f5a1b19b16ffb
I20251028 02:59:58.192831 8553 consensus_queue.cc:237] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.193044 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 0ee59aeedd2c4c9ba94afe6578af649a
I20251028 02:59:58.194356 7960 catalog_manager.cc:5649] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:58.194555 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 81b521c9f57844bb9873f6b26b5cf661 (table default.Table_fd3daefb8df048e7bea5949c30da1e1f [id=5f82b9596e5842deb4d45e19c91629bb]). Expected version 1 got 0
I20251028 02:59:58.183480 8550 ts_tablet_manager.cc:1950] T 3b43da9580244346ae512a7bc9b506d3 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.194995 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 415f127c97114981b32118835d81aa45 (DEFAULT_TABLE table=default.Table_06126295f11f4b82bb6ebec7c701f8ae [id=5e7922fc04a94f05bbdcaf7a4e1437d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.195150 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3b43da9580244346ae512a7bc9b506d3 (table default.Table_a00976e88f0147c5b01560dba597ea82 [id=7244d96a49d34411bb8f9f7416904ae3]) successfully deleted
I20251028 02:59:58.195320 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 415f127c97114981b32118835d81aa45. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.195369 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 8720ea8c68904d57a3fc5c0e94d338ca with delete_type TABLET_DATA_DELETED (replaced by tablet 30232298155542ecb41f5a1b19b16ffb) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.195500 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 0ee59aeedd2c4c9ba94afe6578af649a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:58.195657 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ee1d686a0e4c47f6b214224047a55747 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee1d686a0e4c47f6b214224047a55747 already in progress: creating tablet
W20251028 02:59:58.196048 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet d39008ac6324451a9d1ed54a2ccc3abf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d39008ac6324451a9d1ed54a2ccc3abf already in progress: creating tablet
I20251028 02:59:58.195662 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 02f1a2d489d74e33ab991611fd55966d
W20251028 02:59:58.196444 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet fd8c04efd8c2456b9c7643641c7fbc0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fd8c04efd8c2456b9c7643641c7fbc0d already in progress: creating tablet
I20251028 02:59:58.196611 7964 catalog_manager.cc:6719] ReplaceTablet: tablet 02f1a2d489d74e33ab991611fd55966d deleted and replaced by tablet 5a3394a1e1254df19e6e3f6641de825a
I20251028 02:59:58.196611 8550 tablet_replica.cc:333] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.196763 8550 raft_consensus.cc:2243] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.196892 8550 raft_consensus.cc:2272] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.196962 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 02f1a2d489d74e33ab991611fd55966d with delete_type TABLET_DATA_DELETED (replaced by tablet 5a3394a1e1254df19e6e3f6641de825a) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.197489 8550 ts_tablet_manager.cc:1916] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.198552 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.198966 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 415f127c97114981b32118835d81aa45 (DEFAULT_TABLE table=default.Table_06126295f11f4b82bb6ebec7c701f8ae [id=5e7922fc04a94f05bbdcaf7a4e1437d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.199172 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 99cc3e2b1eef4137bac27c629600f43b (DEFAULT_TABLE table=default.Table_514bd1c4b2974dc28f976754be3f2c1c [id=f8d93791c70842e79584bb358f898287]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.199254 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 415f127c97114981b32118835d81aa45. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.199432 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99cc3e2b1eef4137bac27c629600f43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.202363 8552 ts_tablet_manager.cc:1929] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.202509 8552 log.cc:1199] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/6dfc5240893546d7a6a5397cd08044dd
I20251028 02:59:58.202809 8552 ts_tablet_manager.cc:1950] T 6dfc5240893546d7a6a5397cd08044dd P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.202847 8550 ts_tablet_manager.cc:1929] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.202998 8550 log.cc:1199] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/83059074c365440b84e39cf7a5553eb2
I20251028 02:59:58.203269 8550 ts_tablet_manager.cc:1950] T 83059074c365440b84e39cf7a5553eb2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.203578 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 99cc3e2b1eef4137bac27c629600f43b (DEFAULT_TABLE table=default.Table_514bd1c4b2974dc28f976754be3f2c1c [id=f8d93791c70842e79584bb358f898287]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.203943 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 391608f115e845e88d07ae08ea543e8e (DEFAULT_TABLE table=default.Table_c061797d60f14b298c963858268b5e60 [id=b5dc2cf2c06e4088a2ce80219d6d675f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.204366 8386 tablet_service.cc:1505] Processing CreateTablet for tablet e9857ba66e9e4938abe00b1619a275af (DEFAULT_TABLE table=default.Table_60724e0676e54a429b5cd8d525030e29 [id=3f7bfc9e9f3543f09675bbb31ad2079c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.204672 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9857ba66e9e4938abe00b1619a275af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.206086 8263 tablet_service.cc:1505] Processing CreateTablet for tablet e9857ba66e9e4938abe00b1619a275af (DEFAULT_TABLE table=default.Table_60724e0676e54a429b5cd8d525030e29 [id=3f7bfc9e9f3543f09675bbb31ad2079c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.206282 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.206496 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.206974 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet 8720ea8c68904d57a3fc5c0e94d338ca with delete_type TABLET_DATA_DELETED (replaced by tablet 30232298155542ecb41f5a1b19b16ffb) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.207130 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 8720ea8c68904d57a3fc5c0e94d338ca with delete_type TABLET_DATA_DELETED (replaced by tablet 30232298155542ecb41f5a1b19b16ffb) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.207427 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet 0ee59aeedd2c4c9ba94afe6578af649a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.207602 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 0ee59aeedd2c4c9ba94afe6578af649a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.208057 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet 02f1a2d489d74e33ab991611fd55966d with delete_type TABLET_DATA_DELETED (replaced by tablet 5a3394a1e1254df19e6e3f6641de825a) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.208221 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 02f1a2d489d74e33ab991611fd55966d with delete_type TABLET_DATA_DELETED (replaced by tablet 5a3394a1e1254df19e6e3f6641de825a) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.208758 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_21c8b9b2691f402f931349d798c2b2c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.209013 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 391608f115e845e88d07ae08ea543e8e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:58.209178 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21c8b9b2691f402f931349d798c2b2c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.209411 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 391608f115e845e88d07ae08ea543e8e (DEFAULT_TABLE table=default.Table_c061797d60f14b298c963858268b5e60 [id=b5dc2cf2c06e4088a2ce80219d6d675f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.209653 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 391608f115e845e88d07ae08ea543e8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.210634 9733 tablet.cc:1722] T e7165de68f6c4987a41cd33b5c6298c4 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:58.211781 7964 catalog_manager.cc:5649] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:58.212018 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet b46554b0df894beb8141126aa7ee7b8e (table default.Table_176b16f7b1fd4a30991b82efd4ddd497 [id=9e5ad11a9ea34a3a8c629f48142b68f0]). Expected version 1 got 0
I20251028 02:59:58.212054 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9857ba66e9e4938abe00b1619a275af. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.213780 8551 tablet_replica.cc:333] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.213917 8551 raft_consensus.cc:2243] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.214030 8551 raft_consensus.cc:2272] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.214445 9639 raft_consensus.cc:493] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.214524 9639 raft_consensus.cc:515] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.214941 9639 leader_election.cc:290] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 02:59:58.215646 8356 leader_election.cc:343] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: 6dfc5240893546d7a6a5397cd08044dd
W20251028 02:59:58.215847 8354 leader_election.cc:343] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 6dfc5240893546d7a6a5397cd08044dd
I20251028 02:59:58.215911 8354 leader_election.cc:304] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:58.216089 9639 raft_consensus.cc:2749] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.216513 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99cc3e2b1eef4137bac27c629600f43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.219708 9639 raft_consensus.cc:493] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.219794 9639 raft_consensus.cc:515] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.220114 9639 leader_election.cc:290] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:58.220961 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a808098c70b54ad3a0230352af0b743c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.221594 8545 tablet_bootstrap.cc:492] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.221685 8545 ts_tablet_manager.cc:1403] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.076s user 0.000s sys 0.001s
I20251028 02:59:58.222210 8545 raft_consensus.cc:359] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.222321 8545 raft_consensus.cc:385] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.222363 8545 raft_consensus.cc:740] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.222506 8545 consensus_queue.cc:260] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.222764 8545 ts_tablet_manager.cc:1434] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.218077 8544 tablet_bootstrap.cc:492] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.222960 8545 tablet_bootstrap.cc:492] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.223019 8544 ts_tablet_manager.cc:1403] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.081s user 0.002s sys 0.000s
I20251028 02:59:58.223575 8544 raft_consensus.cc:359] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.223685 8544 raft_consensus.cc:385] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.223716 8544 raft_consensus.cc:740] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.223831 8544 consensus_queue.cc:260] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.224076 8545 tablet_bootstrap.cc:654] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.224160 8544 ts_tablet_manager.cc:1434] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.224370 8544 tablet_bootstrap.cc:492] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.225244 9529 tablet.cc:1722] T e7165de68f6c4987a41cd33b5c6298c4 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:58.225350 8544 tablet_bootstrap.cc:654] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.220708 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a808098c70b54ad3a0230352af0b743c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:58.227806 8145 raft_consensus.cc:2393] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7e4cf694f71438298dd32a346abcf75 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:58.228214 8354 leader_election.cc:304] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:58.229475 9639 raft_consensus.cc:3060] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.230479 9639 raft_consensus.cc:2749] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.232087 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 83059074c365440b84e39cf7a5553eb2 (table default.Table_2b7f5657d1534447a3a29a01698581e1 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]) successfully deleted
W20251028 02:59:58.232568 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 8720ea8c68904d57a3fc5c0e94d338ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8720ea8c68904d57a3fc5c0e94d338ca already in progress: creating tablet
W20251028 02:59:58.233494 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 0ee59aeedd2c4c9ba94afe6578af649a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ee59aeedd2c4c9ba94afe6578af649a already in progress: creating tablet
I20251028 02:59:58.233860 8551 ts_tablet_manager.cc:1916] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.234537 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet f08a6e30211a4d59934c36c7a26ac983 (table default.Table_c62970e81d90490ebe817e6ff5c68b97 [id=b9c53ab5608f49309950ed79b8272710]) successfully deleted
I20251028 02:59:58.234802 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 6dfc5240893546d7a6a5397cd08044dd (table default.Table_c20f674e0d9a4178a66f16d31bc99638 [id=49d0cfc3409f4e2d84b5e1b2634193e5]) successfully deleted
W20251028 02:59:58.234905 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ee1d686a0e4c47f6b214224047a55747 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee1d686a0e4c47f6b214224047a55747 already in progress: creating tablet
W20251028 02:59:58.235020 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet fd8c04efd8c2456b9c7643641c7fbc0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fd8c04efd8c2456b9c7643641c7fbc0d already in progress: creating tablet
W20251028 02:59:58.235114 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet d39008ac6324451a9d1ed54a2ccc3abf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d39008ac6324451a9d1ed54a2ccc3abf already in progress: creating tablet
W20251028 02:59:58.235208 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 already in progress: creating tablet
W20251028 02:59:58.236203 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 8720ea8c68904d57a3fc5c0e94d338ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8720ea8c68904d57a3fc5c0e94d338ca already in progress: creating tablet
W20251028 02:59:58.236349 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 0ee59aeedd2c4c9ba94afe6578af649a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ee59aeedd2c4c9ba94afe6578af649a already in progress: creating tablet
W20251028 02:59:58.236441 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 02f1a2d489d74e33ab991611fd55966d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02f1a2d489d74e33ab991611fd55966d already in progress: creating tablet
I20251028 02:59:58.236691 8545 tablet_bootstrap.cc:492] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.236778 8545 ts_tablet_manager.cc:1403] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20251028 02:59:58.237633 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 529b865389f94471a6849d17ff206e4c (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
I20251028 02:59:58.238013 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4ebe7022d46e41b68350a3b22f24a919" } modify_external_catalogs: true
I20251028 02:59:58.238123 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4ebe7022d46e41b68350a3b22f24a919" } modify_external_catalogs: true
I20251028 02:59:58.238106 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d6b0104aa11a449198e0c05a7f13c6a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.238497 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d6b0104aa11a449198e0c05a7f13c6a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.238502 8546 tablet_bootstrap.cc:492] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.238605 8546 ts_tablet_manager.cc:1403] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.119s user 0.000s sys 0.002s
I20251028 02:59:58.239082 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:58.239151 8546 raft_consensus.cc:359] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.239276 8546 raft_consensus.cc:385] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.239348 8546 raft_consensus.cc:740] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.239500 8546 consensus_queue.cc:260] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.239795 8546 ts_tablet_manager.cc:1434] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:58.240020 8546 tablet_bootstrap.cc:492] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.241052 8546 tablet_bootstrap.cc:654] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.244257 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:58.246399 8545 raft_consensus.cc:359] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.246536 8545 raft_consensus.cc:385] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.246569 8545 raft_consensus.cc:740] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.246707 8545 consensus_queue.cc:260] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.247084 8545 ts_tablet_manager.cc:1434] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20251028 02:59:58.247641 8545 tablet_bootstrap.cc:492] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.249281 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 985a0daa0a544216bbb06b1bc9d19231 (DEFAULT_TABLE table=default.Table_f3795491c9e547af98943428398101b2 [id=8fe996a5bf644b1a9a0d2d3894445a56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.249490 8116 tablet_service.cc:1505] Processing CreateTablet for tablet b2307c69e7ee47cbac31c54be957f5d4 (DEFAULT_TABLE table=default.Table_aee4903456c34845a3085d4f181b13ce [id=ed00bb3f22544235ace2497f3ab154ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.249588 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985a0daa0a544216bbb06b1bc9d19231. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.249825 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2307c69e7ee47cbac31c54be957f5d4. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:58.249907 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 02f1a2d489d74e33ab991611fd55966d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02f1a2d489d74e33ab991611fd55966d already in progress: creating tablet
I20251028 02:59:58.250006 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 55a8f96cd25b4019ae15fad3129dd314 (DEFAULT_TABLE table=default.Table_4ebe7022d46e41b68350a3b22f24a919 [id=54448b53f12f44aba72aa640d0f0abbc]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:58.250330 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 already in progress: creating tablet
I20251028 02:59:58.251543 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55a8f96cd25b4019ae15fad3129dd314. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.252080 8544 tablet_bootstrap.cc:492] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.252158 8544 ts_tablet_manager.cc:1403] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20251028 02:59:58.252537 8545 tablet_bootstrap.cc:654] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.252656 8544 raft_consensus.cc:359] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.252774 8544 raft_consensus.cc:385] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.252808 8544 raft_consensus.cc:740] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.252916 8544 consensus_queue.cc:260] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.253181 8544 ts_tablet_manager.cc:1434] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:58.253386 8544 tablet_bootstrap.cc:492] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.254585 8544 tablet_bootstrap.cc:654] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.255635 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 5a3394a1e1254df19e6e3f6641de825a (DEFAULT_TABLE table=default.Table_6eabdbe1000e4fb6b82c9425cc2e0edd [id=f2b49da1d9f0421f84ff98bc5c05462f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.255903 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 30232298155542ecb41f5a1b19b16ffb (DEFAULT_TABLE table=default.Table_79ced5936022425c91473064b87986d3 [id=375c6744f94644199d8d6f08f21ea021]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.256166 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a3394a1e1254df19e6e3f6641de825a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.257984 8546 tablet_bootstrap.cc:492] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.258085 8546 ts_tablet_manager.cc:1403] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20251028 02:59:58.258649 8546 raft_consensus.cc:359] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.258692 8127 tablet_service.cc:1505] Processing CreateTablet for tablet a7fa9f23989d4bdba6545d0e681c78e5 (DEFAULT_TABLE table=default.Table_03e0732cf61746cb89213075b54e35cc [id=5c59b015d8f14aad8c6d47ec913553b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.258760 8546 raft_consensus.cc:385] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.258796 8546 raft_consensus.cc:740] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.258926 8546 consensus_queue.cc:260] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.259225 8546 ts_tablet_manager.cc:1434] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.259342 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2b7f5657d1534447a3a29a01698581e1" } new_table_name: "default.Table_13dbc4fadcfd41c8945172798836fb62" modify_external_catalogs: true
I20251028 02:59:58.259450 8546 tablet_bootstrap.cc:492] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.260684 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.261412 8113 tablet_service.cc:1505] Processing CreateTablet for tablet e346247505904e819c837c019bc48806 (DEFAULT_TABLE table=default.Table_7b7c98fa38d3422489f2813f75fa7deb [id=fc3bb2aa575043d79d0ab59265040108]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.261760 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30232298155542ecb41f5a1b19b16ffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.261955 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a19abcac1bc24d7aa75a6032b0689b1b" } new_table_name: "default.Table_fe075af668b044a98e7c7128d52890a0" modify_external_catalogs: true
I20251028 02:59:58.263600 8546 tablet_bootstrap.cc:654] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.265120 8249 tablet_service.cc:1505] Processing CreateTablet for tablet a7fa9f23989d4bdba6545d0e681c78e5 (DEFAULT_TABLE table=default.Table_03e0732cf61746cb89213075b54e35cc [id=5c59b015d8f14aad8c6d47ec913553b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.265365 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7fa9f23989d4bdba6545d0e681c78e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.265403 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e346247505904e819c837c019bc48806. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.265506 8419 raft_consensus.cc:1275] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:58.265865 8396 tablet_service.cc:1505] Processing CreateTablet for tablet e346247505904e819c837c019bc48806 (DEFAULT_TABLE table=default.Table_7b7c98fa38d3422489f2813f75fa7deb [id=fc3bb2aa575043d79d0ab59265040108]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.266148 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e346247505904e819c837c019bc48806. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.266570 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7fa9f23989d4bdba6545d0e681c78e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.267006 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 5a3394a1e1254df19e6e3f6641de825a (DEFAULT_TABLE table=default.Table_6eabdbe1000e4fb6b82c9425cc2e0edd [id=f2b49da1d9f0421f84ff98bc5c05462f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.267215 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a3394a1e1254df19e6e3f6641de825a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.267510 8145 raft_consensus.cc:1275] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:58.267563 8256 tablet_service.cc:1505] Processing CreateTablet for tablet b2307c69e7ee47cbac31c54be957f5d4 (DEFAULT_TABLE table=default.Table_aee4903456c34845a3085d4f181b13ce [id=ed00bb3f22544235ace2497f3ab154ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.267787 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2307c69e7ee47cbac31c54be957f5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.267832 8263 tablet_service.cc:1505] Processing CreateTablet for tablet e346247505904e819c837c019bc48806 (DEFAULT_TABLE table=default.Table_7b7c98fa38d3422489f2813f75fa7deb [id=fc3bb2aa575043d79d0ab59265040108]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.268047 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e346247505904e819c837c019bc48806. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.267982 9685 consensus_queue.cc:1048] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:58.268278 9685 consensus_queue.cc:1048] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:58.268653 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 30232298155542ecb41f5a1b19b16ffb (DEFAULT_TABLE table=default.Table_79ced5936022425c91473064b87986d3 [id=375c6744f94644199d8d6f08f21ea021]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.268896 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30232298155542ecb41f5a1b19b16ffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.269068 9685 raft_consensus.cc:493] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.269140 9685 raft_consensus.cc:515] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.269479 9685 leader_election.cc:290] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.269932 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 30232298155542ecb41f5a1b19b16ffb (DEFAULT_TABLE table=default.Table_79ced5936022425c91473064b87986d3 [id=375c6744f94644199d8d6f08f21ea021]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.270073 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d59acfb0e12643ebbb406dbda7bef29b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:58.270141 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30232298155542ecb41f5a1b19b16ffb. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:58.270455 8220 leader_election.cc:343] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.271342 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 985a0daa0a544216bbb06b1bc9d19231 (DEFAULT_TABLE table=default.Table_f3795491c9e547af98943428398101b2 [id=8fe996a5bf644b1a9a0d2d3894445a56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.271553 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985a0daa0a544216bbb06b1bc9d19231. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.271487 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6dde487b1f644e8fb621a40296df4b4d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.271642 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 55a8f96cd25b4019ae15fad3129dd314 (DEFAULT_TABLE table=default.Table_4ebe7022d46e41b68350a3b22f24a919 [id=54448b53f12f44aba72aa640d0f0abbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.271864 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55a8f96cd25b4019ae15fad3129dd314. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:58.271904 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6dde487b1f644e8fb621a40296df4b4d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:58.272838 8218 leader_election.cc:343] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: d59acfb0e12643ebbb406dbda7bef29b
I20251028 02:59:58.272902 8218 leader_election.cc:304] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:58.273113 8553 raft_consensus.cc:2749] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.273715 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 985a0daa0a544216bbb06b1bc9d19231 (DEFAULT_TABLE table=default.Table_f3795491c9e547af98943428398101b2 [id=8fe996a5bf644b1a9a0d2d3894445a56]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.273962 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985a0daa0a544216bbb06b1bc9d19231. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.276294 8384 tablet_service.cc:1505] Processing CreateTablet for tablet 5a3394a1e1254df19e6e3f6641de825a (DEFAULT_TABLE table=default.Table_6eabdbe1000e4fb6b82c9425cc2e0edd [id=f2b49da1d9f0421f84ff98bc5c05462f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.276535 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a3394a1e1254df19e6e3f6641de825a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.278158 8398 tablet_service.cc:1505] Processing CreateTablet for tablet b2307c69e7ee47cbac31c54be957f5d4 (DEFAULT_TABLE table=default.Table_aee4903456c34845a3085d4f181b13ce [id=ed00bb3f22544235ace2497f3ab154ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.278370 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2307c69e7ee47cbac31c54be957f5d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.279069 8386 tablet_service.cc:1505] Processing CreateTablet for tablet a7fa9f23989d4bdba6545d0e681c78e5 (DEFAULT_TABLE table=default.Table_03e0732cf61746cb89213075b54e35cc [id=5c59b015d8f14aad8c6d47ec913553b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.279343 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7fa9f23989d4bdba6545d0e681c78e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.280097 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b552f63895449aea79cbd8867cc063c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.280313 8552 tablet_replica.cc:333] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.280401 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 55a8f96cd25b4019ae15fad3129dd314 (DEFAULT_TABLE table=default.Table_4ebe7022d46e41b68350a3b22f24a919 [id=54448b53f12f44aba72aa640d0f0abbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.280597 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55a8f96cd25b4019ae15fad3129dd314. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.281514 8552 raft_consensus.cc:2243] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.281641 8552 raft_consensus.cc:2272] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.282166 8552 ts_tablet_manager.cc:1916] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.284444 8552 ts_tablet_manager.cc:1929] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.284511 8552 log.cc:1199] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/7b552f63895449aea79cbd8867cc063c
I20251028 02:59:58.284718 8552 ts_tablet_manager.cc:1950] T 7b552f63895449aea79cbd8867cc063c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.285882 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet a808098c70b54ad3a0230352af0b743c (table default.Table_9e950f6a07524cb0b5a0d3d967668ad6 [id=5dd4cb94fa384facbdd1aef677a4b498]). Expected version 1 got 0
I20251028 02:59:58.285969 8544 tablet_bootstrap.cc:492] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.286059 8544 ts_tablet_manager.cc:1403] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.000s
I20251028 02:59:58.286563 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet d59acfb0e12643ebbb406dbda7bef29b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.286892 8552 tablet_replica.cc:333] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.287029 8552 raft_consensus.cc:2243] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.287150 8552 raft_consensus.cc:2272] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.287668 8544 raft_consensus.cc:359] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.287799 8544 raft_consensus.cc:385] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.287834 8544 raft_consensus.cc:740] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.287983 8544 consensus_queue.cc:260] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.288304 8544 ts_tablet_manager.cc:1434] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 02:59:58.288503 8544 tablet_bootstrap.cc:492] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.289057 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 55a8f96cd25b4019ae15fad3129dd314
I20251028 02:59:58.289630 8544 tablet_bootstrap.cc:654] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.290717 8544 tablet_bootstrap.cc:492] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.290807 8544 ts_tablet_manager.cc:1403] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20251028 02:59:58.291285 8544 raft_consensus.cc:359] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.291401 8544 raft_consensus.cc:385] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.291433 8544 raft_consensus.cc:740] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.291549 8544 consensus_queue.cc:260] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.291877 8544 ts_tablet_manager.cc:1434] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:58.292063 8544 tablet_bootstrap.cc:492] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.292505 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 55a8f96cd25b4019ae15fad3129dd314 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.293114 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 55a8f96cd25b4019ae15fad3129dd314 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.293282 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b76abacc8f1b4b3684ac8cb4b6611238"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.293342 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_700c6a40a88a4e53b22b2e376bfbe265"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.293722 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b76abacc8f1b4b3684ac8cb4b6611238 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:58.293720 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_700c6a40a88a4e53b22b2e376bfbe265 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.295377 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 7b552f63895449aea79cbd8867cc063c (table default.Table_8c2ee567a0b74836997f0d500266c256 [id=efaa3797242b4a238c7aac2d7427451c]) successfully deleted
I20251028 02:59:58.296344 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_db269528fac84afd92d7be143be14944"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.296732 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db269528fac84afd92d7be143be14944 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.297474 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 55a8f96cd25b4019ae15fad3129dd314 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:58.297513 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c120a3d56f6b41c28c7f7dcd84359a60 consensus state has no leader
I20251028 02:59:58.297747 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17
I20251028 02:59:58.298027 7960 catalog_manager.cc:6719] ReplaceTablet: tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 deleted and replaced by tablet 751777e4aac9489183e021c2367d3122
W20251028 02:59:58.298266 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6233be1842804618a5efc76a75cd85f5 consensus state has no leader
I20251028 02:59:58.299176 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 with delete_type TABLET_DATA_DELETED (replaced by tablet 751777e4aac9489183e021c2367d3122) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.299713 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 with delete_type TABLET_DATA_DELETED (replaced by tablet 751777e4aac9489183e021c2367d3122) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.300892 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 with delete_type TABLET_DATA_DELETED (replaced by tablet 751777e4aac9489183e021c2367d3122) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:58.303992 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 55a8f96cd25b4019ae15fad3129dd314 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55a8f96cd25b4019ae15fad3129dd314 already in progress: creating tablet
W20251028 02:59:58.304167 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 already in progress: creating tablet
I20251028 02:59:58.304339 8552 ts_tablet_manager.cc:1916] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.306968 8552 ts_tablet_manager.cc:1929] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.307045 8552 log.cc:1199] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/d59acfb0e12643ebbb406dbda7bef29b
I20251028 02:59:58.307278 8552 ts_tablet_manager.cc:1950] T d59acfb0e12643ebbb406dbda7bef29b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.308873 8551 ts_tablet_manager.cc:1929] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.308939 8551 log.cc:1199] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/35d5e3fa27ab46f9823b9cf4f7b3eb2a
I20251028 02:59:58.309111 8551 ts_tablet_manager.cc:1950] T 35d5e3fa27ab46f9823b9cf4f7b3eb2a P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.310298 8544 tablet_bootstrap.cc:654] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.311817 8551 tablet_replica.cc:333] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.311954 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 35d5e3fa27ab46f9823b9cf4f7b3eb2a (table default.Table_1b0dd83e47164c4da9d0b2311dfffc81 [id=6d26ce36ea3a479c9397a1e6ce995d68]) successfully deleted
I20251028 02:59:58.311985 8551 raft_consensus.cc:2243] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.312110 8551 raft_consensus.cc:2272] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:58.312152 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet d39008ac6324451a9d1ed54a2ccc3abf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d39008ac6324451a9d1ed54a2ccc3abf already in progress: creating tablet
W20251028 02:59:58.312297 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet fd8c04efd8c2456b9c7643641c7fbc0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fd8c04efd8c2456b9c7643641c7fbc0d already in progress: creating tablet
W20251028 02:59:58.312428 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ee1d686a0e4c47f6b214224047a55747 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee1d686a0e4c47f6b214224047a55747 already in progress: creating tablet
I20251028 02:59:58.312556 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet d59acfb0e12643ebbb406dbda7bef29b (table default.Table_e4f1369ad6ad4110b5854544d4a266bd [id=ccea14281b2c420a99cede953a8de44d]) successfully deleted
W20251028 02:59:58.312649 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 55a8f96cd25b4019ae15fad3129dd314 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55a8f96cd25b4019ae15fad3129dd314 already in progress: creating tablet
W20251028 02:59:58.312752 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 already in progress: creating tablet
I20251028 02:59:58.313383 8545 tablet_bootstrap.cc:492] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.313475 8545 ts_tablet_manager.cc:1403] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.066s user 0.002s sys 0.000s
I20251028 02:59:58.313688 9551 tablet.cc:1722] T a808098c70b54ad3a0230352af0b743c P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:58.313982 8545 raft_consensus.cc:359] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.314105 8545 raft_consensus.cc:385] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.314139 8545 raft_consensus.cc:740] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.314220 9529 tablet.cc:1722] T a808098c70b54ad3a0230352af0b743c P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:58.314255 8545 consensus_queue.cc:260] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.314508 8545 ts_tablet_manager.cc:1434] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.314684 8545 tablet_bootstrap.cc:492] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.315114 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet a808098c70b54ad3a0230352af0b743c (table default.Table_9e950f6a07524cb0b5a0d3d967668ad6 [id=5dd4cb94fa384facbdd1aef677a4b498]). Expected version 1 got 0
I20251028 02:59:58.315227 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 1de3acf65f744368ab31dc1c82ac554f (table default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc]). Expected version 1 got 0
W20251028 02:59:58.315623 7969 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet 1de3acf65f744368ab31dc1c82ac554f (table default.Table_81a46304107f49a7ba522543a0659d5b [id=739f0673774d472bbb62f176b70998bc], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 1de3acf65f744368ab31dc1c82ac554f consensus state has no leader
I20251028 02:59:58.319396 8545 tablet_bootstrap.cc:654] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.324378 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 874b6660d86b48ebad03ecc54588da28 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d3e422c7ae64b25960714ecc926046a) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.325433 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 4de961f542d84fad8cbce64ce99df114 (DEFAULT_TABLE table=default.Table_21c8b9b2691f402f931349d798c2b2c4 [id=392308a0ef8b4a6aa3d6f281492dd40b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.325781 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4de961f542d84fad8cbce64ce99df114. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.326627 7966 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_9e950f6a07524cb0b5a0d3d967668ad6 [id=5dd4cb94fa384facbdd1aef677a4b498] alter complete (version 1)
I20251028 02:59:58.327530 8544 tablet_bootstrap.cc:492] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.327636 8544 ts_tablet_manager.cc:1403] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.036s user 0.000s sys 0.002s
I20251028 02:59:58.328171 8544 raft_consensus.cc:359] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.328264 8544 raft_consensus.cc:385] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.328295 8544 raft_consensus.cc:740] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.328421 8544 consensus_queue.cc:260] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.328727 8544 ts_tablet_manager.cc:1434] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:58.330271 8544 tablet_bootstrap.cc:492] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.332096 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 4de961f542d84fad8cbce64ce99df114 (DEFAULT_TABLE table=default.Table_21c8b9b2691f402f931349d798c2b2c4 [id=392308a0ef8b4a6aa3d6f281492dd40b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.332401 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4de961f542d84fad8cbce64ce99df114. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.332762 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 4de961f542d84fad8cbce64ce99df114 (DEFAULT_TABLE table=default.Table_21c8b9b2691f402f931349d798c2b2c4 [id=392308a0ef8b4a6aa3d6f281492dd40b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.333022 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4de961f542d84fad8cbce64ce99df114. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.334262 8551 ts_tablet_manager.cc:1916] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.335561 9733 tablet.cc:1722] T a808098c70b54ad3a0230352af0b743c P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:58.336577 8544 tablet_bootstrap.cc:654] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.342674 8546 tablet_bootstrap.cc:492] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.347859 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_74d9969b0d494560bbaa47fec5f802d9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.348318 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74d9969b0d494560bbaa47fec5f802d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.349118 8551 ts_tablet_manager.cc:1929] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.349195 8551 log.cc:1199] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/6dfc5240893546d7a6a5397cd08044dd
I20251028 02:59:58.349402 8551 ts_tablet_manager.cc:1950] T 6dfc5240893546d7a6a5397cd08044dd P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.351984 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_639df418295f428b86043a57b3c6ea58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.352314 8545 tablet_bootstrap.cc:492] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.352316 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3bc315b20a2c41dead9f9a2385b88949"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.352387 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_639df418295f428b86043a57b3c6ea58 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.352417 8545 ts_tablet_manager.cc:1403] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.002s
W20251028 02:59:58.352645 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3bc315b20a2c41dead9f9a2385b88949 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.352970 8545 raft_consensus.cc:359] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.353101 8545 raft_consensus.cc:385] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.353137 8545 raft_consensus.cc:740] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.353204 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fd3daefb8df048e7bea5949c30da1e1f" } new_table_name: "default.Table_c14c90440d424269b7a05b90f49e57e6" modify_external_catalogs: true
I20251028 02:59:58.353253 8545 consensus_queue.cc:260] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.353535 8545 ts_tablet_manager.cc:1434] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:58.353886 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_81a46304107f49a7ba522543a0659d5b" } new_table_name: "default.Table_e5fa1e8f3825450685059a8a45433ef8" modify_external_catalogs: true
I20251028 02:59:58.356690 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bd1f7ca82cb84916aa7e1cbc5d217089" } new_table_name: "default.Table_d2e0c8aaee9749ad947c6697f931ea7d" modify_external_catalogs: true
W20251028 02:59:58.359560 7954 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 6233be1842804618a5efc76a75cd85f5 (table default.Table_fe075af668b044a98e7c7128d52890a0 [id=90acec86b67e49b4ae6ff156698bed74], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6233be1842804618a5efc76a75cd85f5 consensus state has no leader
I20251028 02:59:58.361054 8546 ts_tablet_manager.cc:1403] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.102s user 0.003s sys 0.000s
I20251028 02:59:58.361886 8545 tablet_bootstrap.cc:492] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:58.362082 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1de3acf65f744368ab31dc1c82ac554f consensus state has no leader
I20251028 02:59:58.363051 8545 tablet_bootstrap.cc:654] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.365756 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 5d5c68c16c3e4d37be7f9a505ab44549 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.366284 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet bd50e7d4e28541de9799833c53fe7c02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.366686 8551 tablet_replica.cc:333] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.366815 8551 raft_consensus.cc:2243] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.366905 8551 raft_consensus.cc:2272] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.367427 8551 ts_tablet_manager.cc:1916] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.367398 8546 raft_consensus.cc:359] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.367501 8546 raft_consensus.cc:385] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.367523 8546 raft_consensus.cc:740] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.367635 8546 consensus_queue.cc:260] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.367987 8546 ts_tablet_manager.cc:1434] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20251028 02:59:58.368170 8546 tablet_bootstrap.cc:492] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.369191 8546 tablet_bootstrap.cc:654] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.370301 9401 raft_consensus.cc:493] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.370404 8551 ts_tablet_manager.cc:1929] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.370397 9401 raft_consensus.cc:515] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.370472 8551 log.cc:1199] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/874b6660d86b48ebad03ecc54588da28
I20251028 02:59:58.370682 8551 ts_tablet_manager.cc:1950] T 874b6660d86b48ebad03ecc54588da28 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.370798 9401 leader_election.cc:290] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:58.371120 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "920623f3747046d9be594241e8c13733" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:58.371170 8220 consensus_peers.cc:597] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:58.371434 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "920623f3747046d9be594241e8c13733" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:58.371470 8083 leader_election.cc:343] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:58.371763 8083 leader_election.cc:343] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.371824 8145 raft_consensus.cc:1275] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:58.371829 8083 leader_election.cc:304] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.372331 8562 raft_consensus.cc:2749] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.372283 9595 consensus_queue.cc:1048] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:58.376201 9551 tablet.cc:1722] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:58.376312 9401 raft_consensus.cc:493] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.376382 9401 raft_consensus.cc:515] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.376710 9401 leader_election.cc:290] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.377254 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84c3602bee3b4334a2a30daec986791d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:58.377597 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84c3602bee3b4334a2a30daec986791d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:58.377861 8083 leader_election.cc:343] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.380170 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 81b521c9f57844bb9873f6b26b5cf661 (table default.Table_c14c90440d424269b7a05b90f49e57e6 [id=5f82b9596e5842deb4d45e19c91629bb]). Expected version 2 got 0
I20251028 02:59:58.381108 9529 tablet.cc:1722] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:58.390977 8546 tablet_bootstrap.cc:492] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.391099 8546 ts_tablet_manager.cc:1403] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20251028 02:59:58.391752 8546 raft_consensus.cc:359] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.391891 8546 raft_consensus.cc:385] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.391937 8546 raft_consensus.cc:740] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.392050 8546 consensus_queue.cc:260] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.392355 8546 ts_tablet_manager.cc:1434] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.392519 8546 tablet_bootstrap.cc:492] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.393538 8546 tablet_bootstrap.cc:654] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.394701 8545 tablet_bootstrap.cc:492] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.394914 8545 ts_tablet_manager.cc:1403] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20251028 02:59:58.395483 8545 raft_consensus.cc:359] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.395601 8545 raft_consensus.cc:385] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.395643 8545 raft_consensus.cc:740] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.395799 8545 consensus_queue.cc:260] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:58.395741 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 73a45776f8964e68b79bc973097223a3 consensus state has no leader
I20251028 02:59:58.396091 8545 ts_tablet_manager.cc:1434] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:58.396312 8545 tablet_bootstrap.cc:492] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.398072 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 45572222af854b34a5f26a00f8ad9615 with delete_type TABLET_DATA_DELETED (replaced by tablet d60fc654f6af44b290d46235082a4113) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.398648 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_75f5d65a468e4af6a9b154543d58dce4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.398813 8552 tablet_replica.cc:333] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.398973 8552 raft_consensus.cc:2243] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:58.399072 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75f5d65a468e4af6a9b154543d58dce4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.399098 8552 raft_consensus.cc:2272] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.399833 8552 ts_tablet_manager.cc:1916] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.402362 8552 ts_tablet_manager.cc:1929] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.402433 8552 log.cc:1199] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/45572222af854b34a5f26a00f8ad9615
I20251028 02:59:58.402662 8552 ts_tablet_manager.cc:1950] T 45572222af854b34a5f26a00f8ad9615 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.403342 8545 tablet_bootstrap.cc:654] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.404258 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f3795491c9e547af98943428398101b2" } modify_external_catalogs: true
I20251028 02:59:58.404441 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f3795491c9e547af98943428398101b2" } modify_external_catalogs: true
I20251028 02:59:58.405102 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 6dfc5240893546d7a6a5397cd08044dd (table default.Table_c20f674e0d9a4178a66f16d31bc99638 [id=49d0cfc3409f4e2d84b5e1b2634193e5]) successfully deleted
I20251028 02:59:58.405215 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 9621e42e6a324ecda50fe83c0c53119c from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:58.405287 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 already in progress: creating tablet
W20251028 02:59:58.405380 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 8720ea8c68904d57a3fc5c0e94d338ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8720ea8c68904d57a3fc5c0e94d338ca already in progress: creating tablet
W20251028 02:59:58.405495 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 0ee59aeedd2c4c9ba94afe6578af649a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ee59aeedd2c4c9ba94afe6578af649a already in progress: creating tablet
I20251028 02:59:58.405802 8551 tablet_replica.cc:333] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.405938 8551 raft_consensus.cc:2243] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.406036 8551 raft_consensus.cc:2272] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:58.405598 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 02f1a2d489d74e33ab991611fd55966d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02f1a2d489d74e33ab991611fd55966d already in progress: creating tablet
W20251028 02:59:58.406240 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 55a8f96cd25b4019ae15fad3129dd314 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55a8f96cd25b4019ae15fad3129dd314 already in progress: creating tablet
W20251028 02:59:58.406328 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 already in progress: creating tablet
I20251028 02:59:58.406392 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 874b6660d86b48ebad03ecc54588da28 (table default.Table_c7e0f5beff234f17b69504efc678924d [id=181b50c2d29a4b17b8189e3abb10a420]) successfully deleted
I20251028 02:59:58.406733 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet c0f428e5c6fe43fc8f78761d868ddd09 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:58.407127 8083 leader_election.cc:343] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.407222 8083 leader_election.cc:304] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.407459 9401 raft_consensus.cc:2749] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.409286 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 45572222af854b34a5f26a00f8ad9615 (table default.Table_a387569232e54415a0dd77e906ddcee4 [id=94895dadf0e34a19a454e8755c61c785]) successfully deleted
I20251028 02:59:58.409579 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7b7c98fa38d3422489f2813f75fa7deb" } modify_external_catalogs: true
I20251028 02:59:58.409678 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7b7c98fa38d3422489f2813f75fa7deb" } modify_external_catalogs: true
I20251028 02:59:58.411680 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_479f49bd28d1417a94221c750005fe3f" } modify_external_catalogs: true
I20251028 02:59:58.411816 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_479f49bd28d1417a94221c750005fe3f" } modify_external_catalogs: true
I20251028 02:59:58.414803 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]). Expected version 2 got 0
W20251028 02:59:58.415114 7966 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 240b7da6f46c4c5ab8087ab1dd6275c5 consensus state has no leader
I20251028 02:59:58.416275 8551 ts_tablet_manager.cc:1916] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.418771 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]). Expected version 2 got 0
I20251028 02:59:58.422268 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 985a0daa0a544216bbb06b1bc9d19231
I20251028 02:59:58.423484 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 985a0daa0a544216bbb06b1bc9d19231 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:58.425951 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 985a0daa0a544216bbb06b1bc9d19231 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 985a0daa0a544216bbb06b1bc9d19231 already in progress: creating tablet
I20251028 02:59:58.426208 9401 raft_consensus.cc:493] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.426297 9401 raft_consensus.cc:515] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.426661 9401 leader_election.cc:290] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.427554 8113 tablet_service.cc:1505] Processing CreateTablet for tablet eff471c288544f0fbe4029bb3c52b929 (DEFAULT_TABLE table=default.Table_db269528fac84afd92d7be143be14944 [id=d21f93d0c2b3446db9f2922061806e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.427846 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff471c288544f0fbe4029bb3c52b929. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.428053 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c612bc752847c2be0547cedfd665ae" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:58.428979 8083 leader_election.cc:343] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.430346 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 278fb245128743eb94c5913c8218ab2b (DEFAULT_TABLE table=default.Table_6dde487b1f644e8fb621a40296df4b4d [id=2b1abb6d31664f578c805396c63f11b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.430550 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet df4042c347c24691a0aa0c761b6723c7
I20251028 02:59:58.430598 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 278fb245128743eb94c5913c8218ab2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.431775 7966 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_c14c90440d424269b7a05b90f49e57e6 [id=5f82b9596e5842deb4d45e19c91629bb] alter complete (version 2)
I20251028 02:59:58.432821 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet e346247505904e819c837c019bc48806
I20251028 02:59:58.432876 8125 tablet_service.cc:1505] Processing CreateTablet for tablet fd8b66680c6b4e64b06755270b065db4 (DEFAULT_TABLE table=default.Table_d6b0104aa11a449198e0c05a7f13c6a4 [id=540e433f6373443cbe4b27e4e1120168]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.433132 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8b66680c6b4e64b06755270b065db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.435411 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c612bc752847c2be0547cedfd665ae" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:58.435819 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fa08b2c03e264b18ba4f30628531ed79" } modify_external_catalogs: true
I20251028 02:59:58.435928 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fa08b2c03e264b18ba4f30628531ed79" } modify_external_catalogs: true
W20251028 02:59:58.436573 8083 leader_election.cc:343] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.436653 8083 leader_election.cc:304] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.437204 9401 raft_consensus.cc:2749] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.437379 8111 tablet_service.cc:1505] Processing CreateTablet for tablet ad374b7d30414afa91eb06e439a7081b (DEFAULT_TABLE table=default.Table_700c6a40a88a4e53b22b2e376bfbe265 [id=f4c01ad98fbc416bb42f44fbbbd738e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.437635 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad374b7d30414afa91eb06e439a7081b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.437727 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet df4042c347c24691a0aa0c761b6723c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.438398 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet e346247505904e819c837c019bc48806 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.438799 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 9d6105600c3d42a0a6ac3008bd1a79cc (DEFAULT_TABLE table=default.Table_b76abacc8f1b4b3684ac8cb4b6611238 [id=37602abec8d14917b3a044ede33c5027]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.439050 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d6105600c3d42a0a6ac3008bd1a79cc. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:58.441107 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet df4042c347c24691a0aa0c761b6723c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df4042c347c24691a0aa0c761b6723c7 already in progress: creating tablet
W20251028 02:59:58.441241 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e346247505904e819c837c019bc48806 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e346247505904e819c837c019bc48806 already in progress: creating tablet
I20251028 02:59:58.442950 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4132a2b416cc434fbaebfd7df7e5d878
I20251028 02:59:58.443491 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 4132a2b416cc434fbaebfd7df7e5d878 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.443555 8544 tablet_bootstrap.cc:492] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.443648 8544 ts_tablet_manager.cc:1403] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.113s user 0.000s sys 0.003s
I20251028 02:59:58.444152 8544 raft_consensus.cc:359] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.444262 8544 raft_consensus.cc:385] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.444293 8544 raft_consensus.cc:740] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.444411 8544 consensus_queue.cc:260] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.444733 8544 ts_tablet_manager.cc:1434] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.444952 8544 tablet_bootstrap.cc:492] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.445972 8544 tablet_bootstrap.cc:654] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.443648 8550 tablet_replica.cc:333] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.447402 8550 raft_consensus.cc:2243] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.447518 8550 raft_consensus.cc:2272] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.447867 8550 ts_tablet_manager.cc:1916] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.449278 8551 ts_tablet_manager.cc:1929] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.449352 8551 log.cc:1199] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/bd50e7d4e28541de9799833c53fe7c02
I20251028 02:59:58.449556 8551 ts_tablet_manager.cc:1950] T bd50e7d4e28541de9799833c53fe7c02 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.450119 8550 ts_tablet_manager.cc:1929] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:58.450186 8550 log.cc:1199] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4132a2b416cc434fbaebfd7df7e5d878
I20251028 02:59:58.450248 8545 tablet_bootstrap.cc:492] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.450353 8545 ts_tablet_manager.cc:1403] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.000s
I20251028 02:59:58.450407 9190 raft_consensus.cc:493] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.450428 8550 ts_tablet_manager.cc:1950] T 4132a2b416cc434fbaebfd7df7e5d878 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.450477 9190 raft_consensus.cc:515] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.450867 9190 leader_election.cc:290] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:58.450829 8545 raft_consensus.cc:359] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.450927 8545 raft_consensus.cc:385] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.450959 8545 raft_consensus.cc:740] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.451067 8545 consensus_queue.cc:260] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.451412 8545 ts_tablet_manager.cc:1434] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.451622 8545 tablet_bootstrap.cc:492] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.451973 8551 tablet_replica.cc:333] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.452122 8551 raft_consensus.cc:2243] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.452212 8551 raft_consensus.cc:2272] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.452494 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4132a2b416cc434fbaebfd7df7e5d878 (table default.Table_fa08b2c03e264b18ba4f30628531ed79 [id=4d9d3f97f0d541f7a90885b9ab852d83]) successfully deleted
I20251028 02:59:58.452683 8545 tablet_bootstrap.cc:654] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.452977 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet aaaec176a3884a79a55bb74397a7dc9c with delete_type TABLET_DATA_DELETED (replaced by tablet 2bf100407b034626b4a9f7e2340f9cf0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.453155 8550 tablet_replica.cc:333] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.453284 8550 raft_consensus.cc:2243] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.453380 8550 raft_consensus.cc:2272] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:58.453948 8354 leader_election.cc:343] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: c0f428e5c6fe43fc8f78761d868ddd09
I20251028 02:59:58.455072 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 985a0daa0a544216bbb06b1bc9d19231 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.455353 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 985a0daa0a544216bbb06b1bc9d19231 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.455677 8386 tablet_service.cc:1505] Processing CreateTablet for tablet eff471c288544f0fbe4029bb3c52b929 (DEFAULT_TABLE table=default.Table_db269528fac84afd92d7be143be14944 [id=d21f93d0c2b3446db9f2922061806e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.455679 8261 tablet_service.cc:1505] Processing CreateTablet for tablet eff471c288544f0fbe4029bb3c52b929 (DEFAULT_TABLE table=default.Table_db269528fac84afd92d7be143be14944 [id=d21f93d0c2b3446db9f2922061806e2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.455928 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff471c288544f0fbe4029bb3c52b929. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.455946 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eff471c288544f0fbe4029bb3c52b929. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.458441 8259 tablet_service.cc:1505] Processing CreateTablet for tablet fd8b66680c6b4e64b06755270b065db4 (DEFAULT_TABLE table=default.Table_d6b0104aa11a449198e0c05a7f13c6a4 [id=540e433f6373443cbe4b27e4e1120168]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:58.458577 8356 leader_election.cc:343] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: c0f428e5c6fe43fc8f78761d868ddd09
I20251028 02:59:58.458650 8356 leader_election.cc:304] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:58.458731 8356 raft_consensus.cc:2686] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251028 02:59:58.458694 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 278fb245128743eb94c5913c8218ab2b (DEFAULT_TABLE table=default.Table_6dde487b1f644e8fb621a40296df4b4d [id=2b1abb6d31664f578c805396c63f11b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.458699 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8b66680c6b4e64b06755270b065db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.458804 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 9d6105600c3d42a0a6ac3008bd1a79cc (DEFAULT_TABLE table=default.Table_b76abacc8f1b4b3684ac8cb4b6611238 [id=37602abec8d14917b3a044ede33c5027]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.459038 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d6105600c3d42a0a6ac3008bd1a79cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.459018 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet df4042c347c24691a0aa0c761b6723c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.459650 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 278fb245128743eb94c5913c8218ab2b (DEFAULT_TABLE table=default.Table_6dde487b1f644e8fb621a40296df4b4d [id=2b1abb6d31664f578c805396c63f11b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.459715 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 278fb245128743eb94c5913c8218ab2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.459857 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 278fb245128743eb94c5913c8218ab2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.458815 8387 tablet_service.cc:1505] Processing CreateTablet for tablet fd8b66680c6b4e64b06755270b065db4 (DEFAULT_TABLE table=default.Table_d6b0104aa11a449198e0c05a7f13c6a4 [id=540e433f6373443cbe4b27e4e1120168]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.461272 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd8b66680c6b4e64b06755270b065db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.461635 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet e346247505904e819c837c019bc48806 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.462428 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 4132a2b416cc434fbaebfd7df7e5d878 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.462507 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet bd50e7d4e28541de9799833c53fe7c02 (table default.Table_424dccd707fc4fa1bd9d01e78cab2f92 [id=dbe3cee1ce8a47fab35d8303c2950b1c]) successfully deleted
I20251028 02:59:58.462579 8552 tablet_replica.cc:333] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.462709 8552 raft_consensus.cc:2243] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.462823 8552 raft_consensus.cc:2272] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.463016 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 74120decd53d4de19c5f6fedf70ecaae from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.461071 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet df4042c347c24691a0aa0c761b6723c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.463212 8552 ts_tablet_manager.cc:1916] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.463471 8259 tablet_service.cc:1505] Processing CreateTablet for tablet ad374b7d30414afa91eb06e439a7081b (DEFAULT_TABLE table=default.Table_700c6a40a88a4e53b22b2e376bfbe265 [id=f4c01ad98fbc416bb42f44fbbbd738e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.463685 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad374b7d30414afa91eb06e439a7081b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.465744 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 9d6105600c3d42a0a6ac3008bd1a79cc (DEFAULT_TABLE table=default.Table_b76abacc8f1b4b3684ac8cb4b6611238 [id=37602abec8d14917b3a044ede33c5027]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.465965 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d6105600c3d42a0a6ac3008bd1a79cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.465979 8552 ts_tablet_manager.cc:1929] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:58.466699 8552 log.cc:1199] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4132a2b416cc434fbaebfd7df7e5d878
I20251028 02:59:58.466964 8552 ts_tablet_manager.cc:1950] T 4132a2b416cc434fbaebfd7df7e5d878 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.468487 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_18181568bf2c4744a84690b6dac5f13f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.468919 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_18181568bf2c4744a84690b6dac5f13f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.469820 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet e346247505904e819c837c019bc48806 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.470492 8544 tablet_bootstrap.cc:492] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.470593 8544 ts_tablet_manager.cc:1403] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.001s
I20251028 02:59:58.471117 8544 raft_consensus.cc:359] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.471223 8544 raft_consensus.cc:385] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.471256 8544 raft_consensus.cc:740] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.471421 8544 consensus_queue.cc:260] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.471706 8544 ts_tablet_manager.cc:1434] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.471907 8544 tablet_bootstrap.cc:492] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.472941 8544 tablet_bootstrap.cc:654] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.476624 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 4132a2b416cc434fbaebfd7df7e5d878 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.477440 8386 tablet_service.cc:1505] Processing CreateTablet for tablet ad374b7d30414afa91eb06e439a7081b (DEFAULT_TABLE table=default.Table_700c6a40a88a4e53b22b2e376bfbe265 [id=f4c01ad98fbc416bb42f44fbbbd738e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.477968 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 4902bc3398754802a5c571036cd19fa7 with delete_type TABLET_DATA_DELETED (replaced by tablet b42f039c30a048e299c39bcf44033125) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.478044 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 751777e4aac9489183e021c2367d3122 (DEFAULT_TABLE table=default.Table_71dd113f5d12423d9aca617205523aa1 [id=9cd25ed7b9a04064b29a42f6fcd95e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.478334 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 751777e4aac9489183e021c2367d3122. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.481062 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 74120decd53d4de19c5f6fedf70ecaae
I20251028 02:59:58.481408 7966 catalog_manager.cc:6719] ReplaceTablet: tablet 74120decd53d4de19c5f6fedf70ecaae deleted and replaced by tablet 680f8235c6e64a48bd0009c699760691
I20251028 02:59:58.481483 8550 ts_tablet_manager.cc:1916] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.482137 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 74120decd53d4de19c5f6fedf70ecaae with delete_type TABLET_DATA_DELETED (replaced by tablet 680f8235c6e64a48bd0009c699760691) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.478643 8545 tablet_bootstrap.cc:492] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.482394 8384 tablet_service.cc:1505] Processing CreateTablet for tablet 751777e4aac9489183e021c2367d3122 (DEFAULT_TABLE table=default.Table_71dd113f5d12423d9aca617205523aa1 [id=9cd25ed7b9a04064b29a42f6fcd95e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.482404 8545 ts_tablet_manager.cc:1403] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20251028 02:59:58.482687 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 751777e4aac9489183e021c2367d3122. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.483122 8545 raft_consensus.cc:359] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.483217 8545 raft_consensus.cc:385] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.483246 8545 raft_consensus.cc:740] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.483383 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 7b552f63895449aea79cbd8867cc063c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.483557 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 74120decd53d4de19c5f6fedf70ecaae with delete_type TABLET_DATA_DELETED (replaced by tablet 680f8235c6e64a48bd0009c699760691) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.483397 8545 consensus_queue.cc:260] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.483732 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 74120decd53d4de19c5f6fedf70ecaae with delete_type TABLET_DATA_DELETED (replaced by tablet 680f8235c6e64a48bd0009c699760691) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:58.483899 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 985a0daa0a544216bbb06b1bc9d19231 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 985a0daa0a544216bbb06b1bc9d19231 already in progress: creating tablet
I20251028 02:59:58.483925 8545 ts_tablet_manager.cc:1434] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.483973 8550 ts_tablet_manager.cc:1929] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251028 02:59:58.484081 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet df4042c347c24691a0aa0c761b6723c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df4042c347c24691a0aa0c761b6723c7 already in progress: creating tablet
I20251028 02:59:58.484149 8545 tablet_bootstrap.cc:492] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.484283 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4132a2b416cc434fbaebfd7df7e5d878 (table default.Table_fa08b2c03e264b18ba4f30628531ed79 [id=4d9d3f97f0d541f7a90885b9ab852d83]) successfully deleted
W20251028 02:59:58.484439 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet e346247505904e819c837c019bc48806 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e346247505904e819c837c019bc48806 already in progress: creating tablet
I20251028 02:59:58.485180 8545 tablet_bootstrap.cc:654] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.486512 8552 tablet_replica.cc:333] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.486778 8552 raft_consensus.cc:2243] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:58.487035 8552 raft_consensus.cc:2272] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.487385 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad374b7d30414afa91eb06e439a7081b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.487759 8552 ts_tablet_manager.cc:1916] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.486605 8546 tablet_bootstrap.cc:492] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.488030 8546 ts_tablet_manager.cc:1403] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.096s user 0.002s sys 0.000s
I20251028 02:59:58.488047 8544 tablet_bootstrap.cc:492] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.488131 8544 ts_tablet_manager.cc:1403] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 02:59:58.488610 8544 raft_consensus.cc:359] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.488718 8544 raft_consensus.cc:385] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.488750 8544 raft_consensus.cc:740] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.488888 8544 consensus_queue.cc:260] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.489218 8544 ts_tablet_manager.cc:1434] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.489408 8544 tablet_bootstrap.cc:492] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.490502 8546 raft_consensus.cc:359] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.490598 8546 raft_consensus.cc:385] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.490625 8546 raft_consensus.cc:740] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.491021 8544 tablet_bootstrap.cc:654] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.492092 8546 consensus_queue.cc:260] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.492437 8562 raft_consensus.cc:493] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.492512 8562 raft_consensus.cc:515] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.492870 8562 leader_election.cc:290] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.492915 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 751777e4aac9489183e021c2367d3122 (DEFAULT_TABLE table=default.Table_71dd113f5d12423d9aca617205523aa1 [id=9cd25ed7b9a04064b29a42f6fcd95e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.493173 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 751777e4aac9489183e021c2367d3122. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.495237 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "900973d832c24b62a4554ae9a219cfd8" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.496950 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "900973d832c24b62a4554ae9a219cfd8" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:58.497992 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f06af61617e4be9a753692e9ad4978d with delete_type TABLET_DATA_DELETED (replaced by tablet 26a60624333d43f1b701036b7e9dfdb0) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.498323 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a54d50989a0842bb8cd3b12aca9bf193"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.498785 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a54d50989a0842bb8cd3b12aca9bf193 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.499217 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d2e0c8aaee9749ad947c6697f931ea7d" } new_table_name: "default.Table_d246152b26344aed845d6366d95b40a0" modify_external_catalogs: true
I20251028 02:59:58.500599 9401 raft_consensus.cc:493] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.500700 9401 raft_consensus.cc:515] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
W20251028 02:59:58.500833 8083 leader_election.cc:343] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:58.500981 8083 leader_election.cc:343] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.501045 8083 leader_election.cc:304] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.501068 9401 leader_election.cc:290] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.501212 8562 raft_consensus.cc:2749] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.501494 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73a45776f8964e68b79bc973097223a3" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.502053 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73a45776f8964e68b79bc973097223a3" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:58.502534 8546 ts_tablet_manager.cc:1434] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
W20251028 02:59:58.502754 8083 leader_election.cc:343] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:58.502897 8083 leader_election.cc:343] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.503549 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b001b0dab18e4d0a8a4421ddf1f836d9" } modify_external_catalogs: true
I20251028 02:59:58.503420 8544 tablet_bootstrap.cc:492] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.503675 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b001b0dab18e4d0a8a4421ddf1f836d9" } modify_external_catalogs: true
I20251028 02:59:58.503726 8083 leader_election.cc:304] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.503788 8544 ts_tablet_manager.cc:1403] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20251028 02:59:58.504302 8544 raft_consensus.cc:359] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.504420 8544 raft_consensus.cc:385] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.504452 8544 raft_consensus.cc:740] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.504565 8544 consensus_queue.cc:260] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
W20251028 02:59:58.504834 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 73a45776f8964e68b79bc973097223a3 consensus state has no leader
I20251028 02:59:58.504902 8544 ts_tablet_manager.cc:1434] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.505100 8544 tablet_bootstrap.cc:492] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.506166 8544 tablet_bootstrap.cc:654] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.506953 9401 raft_consensus.cc:2749] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.507148 7966 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 81b521c9f57844bb9873f6b26b5cf661 (table default.Table_c14c90440d424269b7a05b90f49e57e6 [id=5f82b9596e5842deb4d45e19c91629bb]). Expected version 2 got 0
I20251028 02:59:58.512712 7969 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 81b521c9f57844bb9873f6b26b5cf661 (table default.Table_c14c90440d424269b7a05b90f49e57e6 [id=5f82b9596e5842deb4d45e19c91629bb]). Expected version 2 got 0
I20251028 02:59:58.514922 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_129667a395814d7fa62be13abf2feed1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.515180 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f2dee7552f5543b0ac6899b7a28431a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.515559 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_129667a395814d7fa62be13abf2feed1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:58.515650 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2dee7552f5543b0ac6899b7a28431a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.514946 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e0c668202e06476cbbf5c2d3ecda31b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.516639 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0c668202e06476cbbf5c2d3ecda31b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.517271 8552 ts_tablet_manager.cc:1929] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:58.517354 8552 log.cc:1199] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/74120decd53d4de19c5f6fedf70ecaae
I20251028 02:59:58.517622 8552 ts_tablet_manager.cc:1950] T 74120decd53d4de19c5f6fedf70ecaae P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.515014 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d802b3738006457a9d34cd51d2663e92" } new_table_name: "default.Table_927bd3d6ebc54f3aaa7c47e636452f05" modify_external_catalogs: true
I20251028 02:59:58.518791 8546 tablet_bootstrap.cc:492] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.518774 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5d43bd697e934ca4a3b0e56386389091"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.519073 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d43bd697e934ca4a3b0e56386389091 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.520102 8546 tablet_bootstrap.cc:654] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.520406 8550 log.cc:1199] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/aaaec176a3884a79a55bb74397a7dc9c
I20251028 02:59:58.520682 8550 ts_tablet_manager.cc:1950] T aaaec176a3884a79a55bb74397a7dc9c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.521049 8550 tablet_replica.cc:333] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.521193 8550 raft_consensus.cc:2243] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.521299 8550 raft_consensus.cc:2272] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.521785 8550 ts_tablet_manager.cc:1916] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.521906 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet aaaec176a3884a79a55bb74397a7dc9c (table default.Table_3d3d9ba2e032453fbbc6b2635d1b2203 [id=51fb1b3116c14113b2e14ced3ac17262]) successfully deleted
I20251028 02:59:58.522286 8552 tablet_replica.cc:333] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.522699 8551 ts_tablet_manager.cc:1916] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.522957 8552 raft_consensus.cc:2243] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.523164 8552 raft_consensus.cc:2272] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.525632 8551 ts_tablet_manager.cc:1929] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.525972 8551 log.cc:1199] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c0f428e5c6fe43fc8f78761d868ddd09
I20251028 02:59:58.526093 8550 ts_tablet_manager.cc:1929] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.530155 8546 tablet_bootstrap.cc:492] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.530344 8546 ts_tablet_manager.cc:1403] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:58.530879 8546 raft_consensus.cc:359] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.531090 8546 raft_consensus.cc:385] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.531167 8546 raft_consensus.cc:740] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.531356 8550 log.cc:1199] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4902bc3398754802a5c571036cd19fa7
I20251028 02:59:58.531383 8546 consensus_queue.cc:260] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.531713 8550 ts_tablet_manager.cc:1950] T 4902bc3398754802a5c571036cd19fa7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.532140 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4902bc3398754802a5c571036cd19fa7 (table default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]) successfully deleted
I20251028 02:59:58.532459 8550 tablet_replica.cc:333] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.532644 8550 raft_consensus.cc:2243] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.532835 8550 raft_consensus.cc:2272] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.533414 8550 ts_tablet_manager.cc:1916] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.536183 8550 ts_tablet_manager.cc:1929] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:58.536332 8550 log.cc:1199] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/74120decd53d4de19c5f6fedf70ecaae
I20251028 02:59:58.536507 8562 raft_consensus.cc:493] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.536595 8562 raft_consensus.cc:515] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.536973 8562 leader_election.cc:290] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.537122 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]) successfully deleted
I20251028 02:59:58.537683 8550 ts_tablet_manager.cc:1950] T 74120decd53d4de19c5f6fedf70ecaae P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.538090 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f6f58ddaa6940248065a05b75996112" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.542080 8552 ts_tablet_manager.cc:1916] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.542850 8546 ts_tablet_manager.cc:1434] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
W20251028 02:59:58.538656 8083 leader_election.cc:343] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.543821 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]) successfully deleted
I20251028 02:59:58.546658 8546 tablet_bootstrap.cc:492] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.549749 9551 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20251028 02:59:58.550549 8546 tablet_bootstrap.cc:654] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.550742 9529 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20251028 02:59:58.563498 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 06f7d43bea27429fb45413e0dc0ba269
I20251028 02:59:58.564410 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 06f7d43bea27429fb45413e0dc0ba269 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.538407 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f6f58ddaa6940248065a05b75996112" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:58.564961 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 06f7d43bea27429fb45413e0dc0ba269 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06f7d43bea27429fb45413e0dc0ba269 already in progress: creating tablet
I20251028 02:59:58.559702 9733 tablet.cc:1722] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
W20251028 02:59:58.565270 8083 leader_election.cc:343] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.565348 8083 leader_election.cc:304] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.564850 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 06f7d43bea27429fb45413e0dc0ba269 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.565555 8562 raft_consensus.cc:2749] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.565903 8551 ts_tablet_manager.cc:1950] T c0f428e5c6fe43fc8f78761d868ddd09 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.566457 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:58.566627 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet c0f428e5c6fe43fc8f78761d868ddd09 (table default.Table_594bc3babcbf48e4b5d9490f18936563 [id=0ee1306a57fd419497c97f844e58157e]) successfully deleted
I20251028 02:59:58.564265 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 06f7d43bea27429fb45413e0dc0ba269 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.567315 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3355092ddb794533a5545c15b2847d15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.567719 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3355092ddb794533a5545c15b2847d15 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:58.570842 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 985a0daa0a544216bbb06b1bc9d19231 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 985a0daa0a544216bbb06b1bc9d19231 already in progress: creating tablet
I20251028 02:59:58.571965 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_db72e72bf1ce4c70a62bdf4b009366ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.572345 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db72e72bf1ce4c70a62bdf4b009366ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.572817 8545 tablet_bootstrap.cc:492] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.572914 8545 ts_tablet_manager.cc:1403] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.089s user 0.002s sys 0.000s
I20251028 02:59:58.573480 8545 raft_consensus.cc:359] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.573601 8545 raft_consensus.cc:385] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.573635 8545 raft_consensus.cc:740] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.573750 8545 consensus_queue.cc:260] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.574067 8545 ts_tablet_manager.cc:1434] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:58.574263 8545 tablet_bootstrap.cc:492] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.575240 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.576581 8545 tablet_bootstrap.cc:654] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.577158 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:58.582022 8552 ts_tablet_manager.cc:1929] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.582207 8551 tablet_replica.cc:333] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.582015 7960 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_927bd3d6ebc54f3aaa7c47e636452f05 [id=a6ae2be5375e4328848f28f43dfc49ab] alter complete (version 6)
I20251028 02:59:58.582682 8552 log.cc:1199] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3f06af61617e4be9a753692e9ad4978d
W20251028 02:59:58.583012 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet e346247505904e819c837c019bc48806 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e346247505904e819c837c019bc48806 already in progress: creating tablet
W20251028 02:59:58.583199 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet df4042c347c24691a0aa0c761b6723c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df4042c347c24691a0aa0c761b6723c7 already in progress: creating tablet
I20251028 02:59:58.583745 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ad274b5a83e045a685e2d4322bde36fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.584299 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad274b5a83e045a685e2d4322bde36fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.582420 8551 raft_consensus.cc:2243] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:58.585304 8551 raft_consensus.cc:2272] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.585526 8552 ts_tablet_manager.cc:1950] T 3f06af61617e4be9a753692e9ad4978d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.585985 8551 ts_tablet_manager.cc:1916] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.586356 8546 tablet_bootstrap.cc:492] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.586505 8546 ts_tablet_manager.cc:1403] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.001s
I20251028 02:59:58.587056 8546 raft_consensus.cc:359] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.587199 8546 raft_consensus.cc:385] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.587263 8546 raft_consensus.cc:740] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.587519 8546 consensus_queue.cc:260] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.587867 8546 ts_tablet_manager.cc:1434] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.588173 8546 tablet_bootstrap.cc:492] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.589485 8546 tablet_bootstrap.cc:654] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.591689 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_196d7bd17aed4ca7909e5ab6e7653130" } new_table_name: "default.Table_3aeb143d1cf14c2b8217326c4689dd59" modify_external_catalogs: true
W20251028 02:59:58.597476 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 110d3531524a460a8e6c2966ef0c367f consensus state has no leader
I20251028 02:59:58.598215 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet f08a6e30211a4d59934c36c7a26ac983 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.599382 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_dcbff39c1ba04e408b0a0c2ee1f42547" } new_table_name: "default.Table_0b7b98e5884a48de833ab1fbbad97f08" modify_external_catalogs: true
I20251028 02:59:58.599898 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet a19eeac727e142579e34a301a6e2db02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.599983 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3f06af61617e4be9a753692e9ad4978d (table default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3]) successfully deleted
W20251028 02:59:58.600145 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 06f7d43bea27429fb45413e0dc0ba269 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06f7d43bea27429fb45413e0dc0ba269 already in progress: creating tablet
I20251028 02:59:58.600419 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f3ea294303c54da8845388204f1b3a5b" } new_table_name: "default.Table_8231afca62a6444ea97c3daf82c6cde0" modify_external_catalogs: true
I20251028 02:59:58.601789 7967 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]). Expected version 1 got 0
I20251028 02:59:58.602828 8552 tablet_replica.cc:333] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.602990 8552 raft_consensus.cc:2243] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.603120 8552 raft_consensus.cc:2272] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.603768 8545 tablet_bootstrap.cc:492] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.603896 8545 ts_tablet_manager.cc:1403] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20251028 02:59:58.604465 8545 raft_consensus.cc:359] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.604574 8545 raft_consensus.cc:385] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.604604 8545 raft_consensus.cc:740] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.604712 8545 consensus_queue.cc:260] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.604947 8553 raft_consensus.cc:493] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.605043 8553 raft_consensus.cc:515] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.605437 8553 leader_election.cc:290] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.605826 7967 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]). Expected version 1 got 0
I20251028 02:59:58.607096 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 02695b18936d4f928cd55f9b76b90b2e with delete_type TABLET_DATA_DELETED (replaced by tablet ebf449a6d241486a892eba79568c3409) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.607278 8550 tablet_replica.cc:333] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.607436 8550 raft_consensus.cc:2243] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.607450 8552 ts_tablet_manager.cc:1916] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.607542 8550 raft_consensus.cc:2272] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.608129 8550 ts_tablet_manager.cc:1916] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.608896 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_dc0c85038d58432db143e5af2d2f4dab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.609082 8545 ts_tablet_manager.cc:1434] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 02:59:58.609283 8545 tablet_bootstrap.cc:492] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:58.609378 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc0c85038d58432db143e5af2d2f4dab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.611819 8562 raft_consensus.cc:493] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.611922 8562 raft_consensus.cc:515] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:58.611982 8220 leader_election.cc:343] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:58.612257 8544 tablet_bootstrap.cc:492] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.612267 8553 raft_consensus.cc:493] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.612311 8562 leader_election.cc:290] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.612342 8544 ts_tablet_manager.cc:1403] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.107s user 0.002s sys 0.000s
I20251028 02:59:58.612350 8553 raft_consensus.cc:515] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.612663 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1de3acf65f744368ab31dc1c82ac554f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.612721 8553 leader_election.cc:290] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.612804 8544 raft_consensus.cc:359] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.612901 8544 raft_consensus.cc:385] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.612938 8544 raft_consensus.cc:740] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.613068 8544 consensus_queue.cc:260] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
W20251028 02:59:58.613234 8083 leader_election.cc:343] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.613265 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1de3acf65f744368ab31dc1c82ac554f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:58.613390 8544 ts_tablet_manager.cc:1434] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:58.613564 8544 tablet_bootstrap.cc:492] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 02:59:58.613636 8083 leader_election.cc:343] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.613615 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09d65eef81b2421c92678015b6da4fbb" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:58.613693 8083 leader_election.cc:304] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.613734 8416 raft_consensus.cc:2468] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:58.613993 8220 leader_election.cc:304] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:58.614118 8562 raft_consensus.cc:2749] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.614548 8263 tablet_service.cc:1505] Processing CreateTablet for tablet fc82440139b94a5e9ac8120730bed6ba (DEFAULT_TABLE table=default.Table_75f5d65a468e4af6a9b154543d58dce4 [id=acf99bfd8d86425a96ef53f898bfa8d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.614861 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc82440139b94a5e9ac8120730bed6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.615198 8386 tablet_service.cc:1505] Processing CreateTablet for tablet fc82440139b94a5e9ac8120730bed6ba (DEFAULT_TABLE table=default.Table_75f5d65a468e4af6a9b154543d58dce4 [id=acf99bfd8d86425a96ef53f898bfa8d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.615403 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09d65eef81b2421c92678015b6da4fbb" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:58.615495 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc82440139b94a5e9ac8120730bed6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.615506 8144 raft_consensus.cc:2468] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:58.616995 8553 raft_consensus.cc:2804] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:58.617079 8553 raft_consensus.cc:493] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:58.617126 8553 raft_consensus.cc:3060] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.617933 8553 raft_consensus.cc:515] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.618309 8553 leader_election.cc:290] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.618759 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09d65eef81b2421c92678015b6da4fbb" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:58.618882 8416 raft_consensus.cc:3060] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.619835 8416 raft_consensus.cc:2468] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:58.620134 8220 leader_election.cc:304] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:58.620234 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet b993932fd567416984ad6d28f8cadad6 with delete_type TABLET_DATA_DELETED (replaced by tablet 5c1994484abf4b6fbbc3f47527558cdb) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.620666 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "09d65eef81b2421c92678015b6da4fbb" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 02:59:58.620728 8218 leader_election.cc:343] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: fbc0337747a74e20b22aac654c2ab7cc
I20251028 02:59:58.620791 8144 raft_consensus.cc:3060] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.620790 8218 leader_election.cc:304] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:58.621125 9685 raft_consensus.cc:2804] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:58.621199 9685 raft_consensus.cc:697] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:58.621318 9685 consensus_queue.cc:237] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:58.620527 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 58268ac590f3437a942dcafd927a5496 consensus state has no leader
I20251028 02:59:58.621780 8144 raft_consensus.cc:2468] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:58.622057 9685 raft_consensus.cc:2749] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.626129 8545 tablet_bootstrap.cc:654] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:58.627118 8220 consensus_peers.cc:597] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:58.627288 8113 tablet_service.cc:1505] Processing CreateTablet for tablet fc82440139b94a5e9ac8120730bed6ba (DEFAULT_TABLE table=default.Table_75f5d65a468e4af6a9b154543d58dce4 [id=acf99bfd8d86425a96ef53f898bfa8d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.628814 8552 ts_tablet_manager.cc:1929] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.628978 8552 log.cc:1199] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/a19eeac727e142579e34a301a6e2db02
I20251028 02:59:58.629308 8552 ts_tablet_manager.cc:1950] T a19eeac727e142579e34a301a6e2db02 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.633143 7967 catalog_manager.cc:5649] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:58.633390 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:58.634052 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet d59acfb0e12643ebbb406dbda7bef29b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:54 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.635674 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet a19eeac727e142579e34a301a6e2db02 (table default.Table_0ac514a2be6a4e1b904149fd8fdc8eb3 [id=effb15303ea6421a8eebaa32a124678e]) successfully deleted
I20251028 02:59:58.640892 8551 ts_tablet_manager.cc:1929] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 02:59:58.641047 8551 log.cc:1199] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4132a2b416cc434fbaebfd7df7e5d878
I20251028 02:59:58.641392 8551 ts_tablet_manager.cc:1950] T 4132a2b416cc434fbaebfd7df7e5d878 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.641994 8551 tablet_replica.cc:333] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.642138 8551 raft_consensus.cc:2243] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.642227 8551 raft_consensus.cc:2272] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.642522 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4132a2b416cc434fbaebfd7df7e5d878 (table default.Table_fa08b2c03e264b18ba4f30628531ed79 [id=4d9d3f97f0d541f7a90885b9ab852d83]) successfully deleted
I20251028 02:59:58.642697 8551 ts_tablet_manager.cc:1916] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.644989 8551 ts_tablet_manager.cc:1929] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.645078 8551 log.cc:1199] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/7b552f63895449aea79cbd8867cc063c
I20251028 02:59:58.645268 8551 ts_tablet_manager.cc:1950] T 7b552f63895449aea79cbd8867cc063c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.649780 8550 ts_tablet_manager.cc:1929] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.650301 8545 tablet_bootstrap.cc:492] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.650394 8545 ts_tablet_manager.cc:1403] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20251028 02:59:58.650122 8111 tablet_service.cc:1505] Processing CreateTablet for tablet f13374a1b255403e9df82157903a9a0e (DEFAULT_TABLE table=default.Table_74d9969b0d494560bbaa47fec5f802d9 [id=6fce38582683440c80d05b17c558ed35]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.650866 8545 raft_consensus.cc:359] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.650966 8545 raft_consensus.cc:385] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.651016 8545 raft_consensus.cc:740] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.650723 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc82440139b94a5e9ac8120730bed6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.651165 8545 consensus_queue.cc:260] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.651892 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 763022e76e23448a8ae0c9320eb8fffc (DEFAULT_TABLE table=default.Table_639df418295f428b86043a57b3c6ea58 [id=fbf00393ff6f45f8aac717b24d3c5f1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.652117 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763022e76e23448a8ae0c9320eb8fffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.653860 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f13374a1b255403e9df82157903a9a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.651672 8545 ts_tablet_manager.cc:1434] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.656128 9529 tablet.cc:1722] T 5fd8682b26c544d58c79f22dd469eedc P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:58.657325 9551 tablet.cc:1722] T 5fd8682b26c544d58c79f22dd469eedc P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:58.661188 9733 tablet.cc:1722] T 5fd8682b26c544d58c79f22dd469eedc P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:58.661712 9190 raft_consensus.cc:493] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.661790 9190 raft_consensus.cc:515] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.662147 9190 leader_election.cc:290] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:58.663492 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81b521c9f57844bb9873f6b26b5cf661" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.666600 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "81b521c9f57844bb9873f6b26b5cf661" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:58.667141 8354 leader_election.cc:304] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:58.667608 9639 raft_consensus.cc:2749] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.670073 8544 tablet_bootstrap.cc:654] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.671140 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 37f2c8db834d4e3e93af1b5e40f1c71e (DEFAULT_TABLE table=default.Table_3bc315b20a2c41dead9f9a2385b88949 [id=10c6d769e349499da394f34403d5d63d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.671438 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37f2c8db834d4e3e93af1b5e40f1c71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.678664 8545 tablet_bootstrap.cc:492] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.679832 8545 tablet_bootstrap.cc:654] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.680855 7967 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_8231afca62a6444ea97c3daf82c6cde0 [id=0cd346d2b86d4c2494a2289ae312dca7] alter complete (version 1)
I20251028 02:59:58.683362 8550 log.cc:1199] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/02695b18936d4f928cd55f9b76b90b2e
I20251028 02:59:58.683686 8550 ts_tablet_manager.cc:1950] T 02695b18936d4f928cd55f9b76b90b2e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.685324 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 02695b18936d4f928cd55f9b76b90b2e (table default.Table_e2683e2b014947bd87e048f63aa666f1 [id=058a75d461a0497a9268f8edf3506ece]) successfully deleted
I20251028 02:59:58.688166 8550 tablet_replica.cc:333] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.688412 8550 raft_consensus.cc:2243] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.688746 8550 raft_consensus.cc:2272] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.688970 8551 tablet_replica.cc:333] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.689167 8551 raft_consensus.cc:2243] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.689324 8551 raft_consensus.cc:2272] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.689951 8263 tablet_service.cc:1505] Processing CreateTablet for tablet f13374a1b255403e9df82157903a9a0e (DEFAULT_TABLE table=default.Table_74d9969b0d494560bbaa47fec5f802d9 [id=6fce38582683440c80d05b17c558ed35]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.690063 8550 ts_tablet_manager.cc:1916] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.690132 9190 raft_consensus.cc:493] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.690264 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f13374a1b255403e9df82157903a9a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.690218 9190 raft_consensus.cc:515] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.690606 9190 leader_election.cc:290] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:58.691085 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "322cbd8253ce45118ad14e8f8f2f1f74" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:58.691208 8145 raft_consensus.cc:2468] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:58.691224 8551 ts_tablet_manager.cc:1916] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.691444 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "322cbd8253ce45118ad14e8f8f2f1f74" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.691530 8354 leader_election.cc:304] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:58.691560 8271 raft_consensus.cc:2468] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:58.691685 9190 raft_consensus.cc:2804] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:58.691731 9190 raft_consensus.cc:493] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:58.691776 9190 raft_consensus.cc:3060] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.692799 9190 raft_consensus.cc:515] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.693207 9190 leader_election.cc:290] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:58.693542 8545 tablet_bootstrap.cc:492] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.693630 8545 ts_tablet_manager.cc:1403] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20251028 02:59:58.693737 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "322cbd8253ce45118ad14e8f8f2f1f74" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:58.693851 8271 raft_consensus.cc:3060] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.694197 8545 raft_consensus.cc:359] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.694300 8545 raft_consensus.cc:385] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.694331 8545 raft_consensus.cc:740] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.694442 8545 consensus_queue.cc:260] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.694739 8545 ts_tablet_manager.cc:1434] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:58.694857 8271 raft_consensus.cc:2468] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:58.695088 8545 tablet_bootstrap.cc:492] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.695107 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "322cbd8253ce45118ad14e8f8f2f1f74" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:58.695225 8145 raft_consensus.cc:3060] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.696264 8145 raft_consensus.cc:2468] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:58.697189 8354 leader_election.cc:304] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:58.698716 8550 ts_tablet_manager.cc:1929] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.699362 8386 tablet_service.cc:1505] Processing CreateTablet for tablet f13374a1b255403e9df82157903a9a0e (DEFAULT_TABLE table=default.Table_74d9969b0d494560bbaa47fec5f802d9 [id=6fce38582683440c80d05b17c558ed35]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.699632 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f13374a1b255403e9df82157903a9a0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.700371 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet b46554b0df894beb8141126aa7ee7b8e (table default.Table_176b16f7b1fd4a30991b82efd4ddd497 [id=9e5ad11a9ea34a3a8c629f48142b68f0]). Expected version 1 got 0
I20251028 02:59:58.702519 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 763022e76e23448a8ae0c9320eb8fffc (DEFAULT_TABLE table=default.Table_639df418295f428b86043a57b3c6ea58 [id=fbf00393ff6f45f8aac717b24d3c5f1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.702790 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763022e76e23448a8ae0c9320eb8fffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.705328 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet f6ac7b5fff0848cebf710ba3ed50def4 with delete_type TABLET_DATA_DELETED (replaced by tablet 4f90e3c3524e4498aa6c4bfc5304d44b) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.705834 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 763022e76e23448a8ae0c9320eb8fffc (DEFAULT_TABLE table=default.Table_639df418295f428b86043a57b3c6ea58 [id=fbf00393ff6f45f8aac717b24d3c5f1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.705935 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 37f2c8db834d4e3e93af1b5e40f1c71e (DEFAULT_TABLE table=default.Table_3bc315b20a2c41dead9f9a2385b88949 [id=10c6d769e349499da394f34403d5d63d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.706128 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763022e76e23448a8ae0c9320eb8fffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.706185 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 37f2c8db834d4e3e93af1b5e40f1c71e (DEFAULT_TABLE table=default.Table_3bc315b20a2c41dead9f9a2385b88949 [id=10c6d769e349499da394f34403d5d63d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.706264 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37f2c8db834d4e3e93af1b5e40f1c71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.706396 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37f2c8db834d4e3e93af1b5e40f1c71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.706583 9190 raft_consensus.cc:2804] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:58.706667 9190 raft_consensus.cc:697] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:58.706789 9190 consensus_queue.cc:237] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.711818 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet b46554b0df894beb8141126aa7ee7b8e (table default.Table_176b16f7b1fd4a30991b82efd4ddd497 [id=9e5ad11a9ea34a3a8c629f48142b68f0]). Expected version 1 got 0
I20251028 02:59:58.715569 8545 tablet_bootstrap.cc:654] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.718561 8546 tablet_bootstrap.cc:492] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.718680 8546 ts_tablet_manager.cc:1403] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.131s user 0.002s sys 0.000s
I20251028 02:59:58.719172 8546 raft_consensus.cc:359] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.719290 8546 raft_consensus.cc:385] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.719355 8546 raft_consensus.cc:740] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.719475 8546 consensus_queue.cc:260] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.719750 8546 ts_tablet_manager.cc:1434] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.719962 8546 tablet_bootstrap.cc:492] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.720904 8546 tablet_bootstrap.cc:654] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.708388 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 7b552f63895449aea79cbd8867cc063c (table default.Table_8c2ee567a0b74836997f0d500266c256 [id=efaa3797242b4a238c7aac2d7427451c]) successfully deleted
I20251028 02:59:58.723903 8550 log.cc:1199] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/b993932fd567416984ad6d28f8cadad6
I20251028 02:59:58.724273 8550 ts_tablet_manager.cc:1950] T b993932fd567416984ad6d28f8cadad6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.724831 7964 catalog_manager.cc:5649] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:58.726100 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6e2c063c779e4b639714776f91d8e559" } modify_external_catalogs: true
I20251028 02:59:58.726267 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6e2c063c779e4b639714776f91d8e559" } modify_external_catalogs: true
I20251028 02:59:58.727285 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 0cb76fbe3b2446c7bfc1aa23e8be2513 (DEFAULT_TABLE table=default.Table_a54d50989a0842bb8cd3b12aca9bf193 [id=185e21a2b9074adfb0e22c101e698c1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.728081 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cb76fbe3b2446c7bfc1aa23e8be2513. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.728906 8416 raft_consensus.cc:3060] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.727587 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 38ad9a5b7f0d46d29746d7ed1f8ffda2 (DEFAULT_TABLE table=default.Table_dc0c85038d58432db143e5af2d2f4dab [id=a4323e2b40ba4ddabfd54089fe928c23]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.730861 9733 tablet.cc:1722] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:58.730958 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ad9a5b7f0d46d29746d7ed1f8ffda2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.731333 8127 tablet_service.cc:1505] Processing CreateTablet for tablet a52b2fe9282b486682a961fc2136557c (DEFAULT_TABLE table=default.Table_e0c668202e06476cbbf5c2d3ecda31b3 [id=d2a2c394e6434445abdeb95f4554ab4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.731384 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 848dbb23d5604a7085947771c5ee3d5e (DEFAULT_TABLE table=default.Table_f2dee7552f5543b0ac6899b7a28431a2 [id=715f093c061941709983b951959cb8c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.731542 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a52b2fe9282b486682a961fc2136557c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.731578 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 2a57c473fcd74b92819105d83f52b3a2 (DEFAULT_TABLE table=default.Table_ad274b5a83e045a685e2d4322bde36fb [id=b008d7a39fb54a45a6b42872a0bead68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.731783 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 848dbb23d5604a7085947771c5ee3d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.731905 8112 tablet_service.cc:1505] Processing CreateTablet for tablet e3b35a5a341641738a9e6a719d6baa35 (DEFAULT_TABLE table=default.Table_3355092ddb794533a5545c15b2847d15 [id=3d48f7f5019a449ca4e5d21bd2fca7f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.732105 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3b35a5a341641738a9e6a719d6baa35. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.733525 8128 tablet_service.cc:1505] Processing CreateTablet for tablet d52cd7ecd9c741d2a59e06396b5b384a (DEFAULT_TABLE table=default.Table_5d43bd697e934ca4a3b0e56386389091 [id=98bbb904e86a4ea28a1cccaf618d62c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.733712 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d52cd7ecd9c741d2a59e06396b5b384a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.733865 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 680f8235c6e64a48bd0009c699760691 (DEFAULT_TABLE table=default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.734061 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 680f8235c6e64a48bd0009c699760691. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.735653 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 46616e3a5c934d698a097fa1565a4a08 (DEFAULT_TABLE table=default.Table_129667a395814d7fa62be13abf2feed1 [id=cc97298f5e594d2487b31cfb0af17186]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.735848 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46616e3a5c934d698a097fa1565a4a08. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.736024 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a57c473fcd74b92819105d83f52b3a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.736166 8122 tablet_service.cc:1505] Processing CreateTablet for tablet af23fe1145a54b76a9f8e7249b87ca0d (DEFAULT_TABLE table=default.Table_18181568bf2c4744a84690b6dac5f13f [id=eb8a8d39e6f84bde9ed20d3c510baa24]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.736352 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af23fe1145a54b76a9f8e7249b87ca0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.737385 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_12b6b08740be470382334a74e5df73cb" } modify_external_catalogs: true
I20251028 02:59:58.737509 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_12b6b08740be470382334a74e5df73cb" } modify_external_catalogs: true
I20251028 02:59:58.738893 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet b993932fd567416984ad6d28f8cadad6 (table default.Table_5d25b16481c84800a7dbb53c61132cbb [id=b0407c8f78a84245a954c67e888da3ec]) successfully deleted
I20251028 02:59:58.739174 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2681fbe951fb4a88bbf4c192aac594d1" } modify_external_catalogs: true
I20251028 02:59:58.739270 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2681fbe951fb4a88bbf4c192aac594d1" } modify_external_catalogs: true
I20251028 02:59:58.742563 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 6455fca4d6684904a255751ce63258f0 (DEFAULT_TABLE table=default.Table_db72e72bf1ce4c70a62bdf4b009366ac [id=1a036b31b861428d909411cd65fdc546]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.742846 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6455fca4d6684904a255751ce63258f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.747414 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 746eae30ca284758a6e3ed8ad80c2cdc
I20251028 02:59:58.748330 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 12d103e9b5024a87aa232c26e0c51ab9
I20251028 02:59:58.748401 8551 ts_tablet_manager.cc:1929] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:58.748476 8551 log.cc:1199] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/74120decd53d4de19c5f6fedf70ecaae
I20251028 02:59:58.748718 8551 ts_tablet_manager.cc:1950] T 74120decd53d4de19c5f6fedf70ecaae P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.751025 8546 tablet_bootstrap.cc:492] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.751070 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 746eae30ca284758a6e3ed8ad80c2cdc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.751134 8546 ts_tablet_manager.cc:1403] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20251028 02:59:58.751660 8546 raft_consensus.cc:359] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.751765 8546 raft_consensus.cc:385] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.751793 8546 raft_consensus.cc:740] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.751904 8546 consensus_queue.cc:260] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.752166 8546 ts_tablet_manager.cc:1434] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.752360 8546 tablet_bootstrap.cc:492] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.753240 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 12d103e9b5024a87aa232c26e0c51ab9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.758179 8546 tablet_bootstrap.cc:654] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.762924 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3b0c593a3b96434db9ac42d260fc6997
I20251028 02:59:58.763821 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b0c593a3b96434db9ac42d260fc6997 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.763973 8546 tablet_bootstrap.cc:492] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.764083 8546 ts_tablet_manager.cc:1403] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:58.764595 8546 raft_consensus.cc:359] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.764710 8546 raft_consensus.cc:385] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.764746 8546 raft_consensus.cc:740] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.764920 8546 consensus_queue.cc:260] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.765411 8546 ts_tablet_manager.cc:1434] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.765800 8546 tablet_bootstrap.cc:492] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.766741 8546 tablet_bootstrap.cc:654] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.771612 9190 raft_consensus.cc:493] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.771718 9190 raft_consensus.cc:515] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.772069 9190 leader_election.cc:290] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 02:59:58.772922 8354 leader_election.cc:343] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: f08a6e30211a4d59934c36c7a26ac983
W20251028 02:59:58.776810 8356 leader_election.cc:343] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: f08a6e30211a4d59934c36c7a26ac983
I20251028 02:59:58.776895 8356 leader_election.cc:304] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:58.777400 9190 raft_consensus.cc:2749] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.779596 8545 tablet_bootstrap.cc:492] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.779695 8545 ts_tablet_manager.cc:1403] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.085s user 0.002s sys 0.000s
I20251028 02:59:58.779778 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1f11d33cf9ac4f11bc540fa516f55d2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.780157 8545 raft_consensus.cc:359] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.780246 8545 raft_consensus.cc:385] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251028 02:59:58.780231 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f11d33cf9ac4f11bc540fa516f55d2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.780274 8545 raft_consensus.cc:740] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.780382 8545 consensus_queue.cc:260] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.780632 8545 ts_tablet_manager.cc:1434] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:58.780812 8545 tablet_bootstrap.cc:492] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.781132 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet d60fc654f6af44b290d46235082a4113 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.781805 8545 tablet_bootstrap.cc:654] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.782765 8553 raft_consensus.cc:493] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.782856 8553 raft_consensus.cc:515] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.783208 8553 leader_election.cc:290] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.784320 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "920623f3747046d9be594241e8c13733" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:58.784344 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "920623f3747046d9be594241e8c13733" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:58.784462 8145 raft_consensus.cc:2468] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
W20251028 02:59:58.784663 8220 leader_election.cc:343] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.784888 8218 leader_election.cc:304] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.785079 8553 raft_consensus.cc:2804] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:58.785120 8553 raft_consensus.cc:493] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:58.785152 8553 raft_consensus.cc:3060] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.786123 8553 raft_consensus.cc:515] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.786439 8553 leader_election.cc:290] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.786907 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d60fc654f6af44b290d46235082a4113
I20251028 02:59:58.787073 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "920623f3747046d9be594241e8c13733" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:58.787181 7969 catalog_manager.cc:6719] ReplaceTablet: tablet d60fc654f6af44b290d46235082a4113 deleted and replaced by tablet 0d42a5987fa540ce86afdbbe1922896e
I20251028 02:59:58.787185 8145 raft_consensus.cc:3060] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.788254 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "920623f3747046d9be594241e8c13733" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:58.788360 8145 raft_consensus.cc:2468] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
W20251028 02:59:58.788594 8220 leader_election.cc:343] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.788906 8218 leader_election.cc:304] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.789969 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet d60fc654f6af44b290d46235082a4113 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d42a5987fa540ce86afdbbe1922896e) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.791261 8553 raft_consensus.cc:2804] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:58.791414 8553 raft_consensus.cc:697] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:58.791553 8553 consensus_queue.cc:237] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.792896 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet b9803f4cfb9b4ef9a1954c83985d6258 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.795974 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d39b99f0145b444a926eb0ecda4662e6" } modify_external_catalogs: true
I20251028 02:59:58.796180 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d39b99f0145b444a926eb0ecda4662e6" } modify_external_catalogs: true
I20251028 02:59:58.801298 9595 consensus_queue.cc:1048] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:58.802912 8551 tablet_replica.cc:333] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.803148 8551 raft_consensus.cc:2243] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.803359 8551 raft_consensus.cc:2272] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.805532 8551 ts_tablet_manager.cc:1916] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.805526 9788 raft_consensus.cc:493] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.805621 9788 raft_consensus.cc:515] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.806066 9788 leader_election.cc:290] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.806993 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "240b7da6f46c4c5ab8087ab1dd6275c5" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.807317 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "240b7da6f46c4c5ab8087ab1dd6275c5" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:58.807624 8083 leader_election.cc:343] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:58.807754 8083 leader_election.cc:343] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.807803 8083 leader_election.cc:304] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.808135 9401 raft_consensus.cc:2749] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.808885 8551 ts_tablet_manager.cc:1929] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.809356 8545 tablet_bootstrap.cc:492] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.809471 8545 ts_tablet_manager.cc:1403] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.001s
I20251028 02:59:58.809948 8545 raft_consensus.cc:359] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.810043 8545 raft_consensus.cc:385] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.810071 8545 raft_consensus.cc:740] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.810182 8545 consensus_queue.cc:260] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.810428 8545 ts_tablet_manager.cc:1434] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.810595 8545 tablet_bootstrap.cc:492] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.812186 8545 tablet_bootstrap.cc:654] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:58.813805 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 746eae30ca284758a6e3ed8ad80c2cdc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 746eae30ca284758a6e3ed8ad80c2cdc already in progress: creating tablet
I20251028 02:59:58.815465 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet b46554b0df894beb8141126aa7ee7b8e (table default.Table_176b16f7b1fd4a30991b82efd4ddd497 [id=9e5ad11a9ea34a3a8c629f48142b68f0]). Expected version 1 got 0
I20251028 02:59:58.815900 8545 tablet_bootstrap.cc:492] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.816031 8545 ts_tablet_manager.cc:1403] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20251028 02:59:58.816583 8545 raft_consensus.cc:359] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.816723 8545 raft_consensus.cc:385] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.816789 8545 raft_consensus.cc:740] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.816931 8545 consensus_queue.cc:260] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.817248 8545 ts_tablet_manager.cc:1434] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:58.817436 8545 tablet_bootstrap.cc:492] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.818409 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9b92443ae55f4a489d92d6175b03f377
I20251028 02:59:58.818522 8545 tablet_bootstrap.cc:654] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.820087 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 0d42a5987fa540ce86afdbbe1922896e (DEFAULT_TABLE table=default.Table_a387569232e54415a0dd77e906ddcee4 [id=94895dadf0e34a19a454e8755c61c785]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.820379 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d42a5987fa540ce86afdbbe1922896e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.818871 7963 catalog_manager.cc:5649] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:58.829602 8551 log.cc:1199] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/f08a6e30211a4d59934c36c7a26ac983
I20251028 02:59:58.830106 8551 ts_tablet_manager.cc:1950] T f08a6e30211a4d59934c36c7a26ac983 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.830982 8551 tablet_replica.cc:333] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
W20251028 02:59:58.827972 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 12d103e9b5024a87aa232c26e0c51ab9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12d103e9b5024a87aa232c26e0c51ab9 already in progress: creating tablet
I20251028 02:59:58.831336 8551 raft_consensus.cc:2243] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.831485 8551 raft_consensus.cc:2272] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:58.831585 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 3b0c593a3b96434db9ac42d260fc6997 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b0c593a3b96434db9ac42d260fc6997 already in progress: creating tablet
W20251028 02:59:58.831787 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet d60fc654f6af44b290d46235082a4113 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d60fc654f6af44b290d46235082a4113 already in progress: creating tablet
I20251028 02:59:58.832178 8551 ts_tablet_manager.cc:1916] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.836130 8546 tablet_bootstrap.cc:492] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.836232 8546 ts_tablet_manager.cc:1403] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.070s user 0.002s sys 0.000s
I20251028 02:59:58.836687 8546 raft_consensus.cc:359] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.836793 8546 raft_consensus.cc:385] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.836825 8546 raft_consensus.cc:740] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.836880 8544 tablet_bootstrap.cc:492] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.836941 8546 consensus_queue.cc:260] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.836982 8544 ts_tablet_manager.cc:1403] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.223s user 0.003s sys 0.001s
I20251028 02:59:58.837231 8546 ts_tablet_manager.cc:1434] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.837410 8546 tablet_bootstrap.cc:492] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.838358 8546 tablet_bootstrap.cc:654] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.842074 8551 ts_tablet_manager.cc:1929] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.842584 8551 log.cc:1199] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/d59acfb0e12643ebbb406dbda7bef29b
I20251028 02:59:58.842465 8544 raft_consensus.cc:359] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.842849 8546 tablet_bootstrap.cc:492] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.842911 8544 raft_consensus.cc:385] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.842947 8546 ts_tablet_manager.cc:1403] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20251028 02:59:58.842967 8544 raft_consensus.cc:740] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.843123 8544 consensus_queue.cc:260] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.843526 8546 raft_consensus.cc:359] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.843623 8546 raft_consensus.cc:385] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.843662 8546 raft_consensus.cc:740] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.843791 8545 tablet_bootstrap.cc:492] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.843783 8546 consensus_queue.cc:260] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.843866 8545 ts_tablet_manager.cc:1403] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.000s
I20251028 02:59:58.844080 8546 ts_tablet_manager.cc:1434] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:58.844254 8546 tablet_bootstrap.cc:492] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.844380 8545 raft_consensus.cc:359] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.844470 8545 raft_consensus.cc:385] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.844499 8545 raft_consensus.cc:740] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.844612 8545 consensus_queue.cc:260] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.844907 8545 ts_tablet_manager.cc:1434] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.845139 8545 tablet_bootstrap.cc:492] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.845474 8544 ts_tablet_manager.cc:1434] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20251028 02:59:58.845623 8546 tablet_bootstrap.cc:654] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.845680 8544 tablet_bootstrap.cc:492] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.846978 8544 tablet_bootstrap.cc:654] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.851990 9595 raft_consensus.cc:493] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.852114 9595 raft_consensus.cc:515] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.852519 9595 leader_election.cc:290] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.853617 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93ef23900c0e4498934208352eb706f8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:58.854092 8220 leader_election.cc:343] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.854898 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93ef23900c0e4498934208352eb706f8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:58.855041 8144 raft_consensus.cc:2468] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:58.855383 8218 leader_election.cc:304] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:58.855562 9595 raft_consensus.cc:2804] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:58.855628 9595 raft_consensus.cc:493] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:58.855669 9595 raft_consensus.cc:3060] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.856885 9595 raft_consensus.cc:515] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.857035 8545 tablet_bootstrap.cc:654] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.857254 9595 leader_election.cc:290] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.857571 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93ef23900c0e4498934208352eb706f8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:58.857713 8144 raft_consensus.cc:3060] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.857842 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b92443ae55f4a489d92d6175b03f377 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.857956 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93ef23900c0e4498934208352eb706f8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:58.858031 8550 tablet_replica.cc:333] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.858186 8550 raft_consensus.cc:2243] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20251028 02:59:58.858289 8220 leader_election.cc:343] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.858317 8550 raft_consensus.cc:2272] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.858551 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_700c6a40a88a4e53b22b2e376bfbe265" } new_table_name: "default.Table_fbc05387d417457493c6bcf1a472ed74" modify_external_catalogs: true
I20251028 02:59:58.858742 8550 ts_tablet_manager.cc:1916] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.858817 8144 raft_consensus.cc:2468] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:58.859076 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bcfa6c66d227460abd8df8d84c443a3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.859215 8551 ts_tablet_manager.cc:1950] T d59acfb0e12643ebbb406dbda7bef29b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.859202 8218 leader_election.cc:304] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
W20251028 02:59:58.859503 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcfa6c66d227460abd8df8d84c443a3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.859669 9595 raft_consensus.cc:2804] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:58.859822 9595 raft_consensus.cc:697] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:58.860051 9595 consensus_queue.cc:237] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.860105 8551 tablet_replica.cc:333] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.860241 8551 raft_consensus.cc:2243] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.860335 8551 raft_consensus.cc:2272] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.861140 8550 ts_tablet_manager.cc:1929] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:58.861140 8551 ts_tablet_manager.cc:1916] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.861210 8550 log.cc:1199] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9b92443ae55f4a489d92d6175b03f377
I20251028 02:59:58.861452 8550 ts_tablet_manager.cc:1950] T 9b92443ae55f4a489d92d6175b03f377 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.861827 9190 raft_consensus.cc:493] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.861898 9190 raft_consensus.cc:515] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.862049 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9b92443ae55f4a489d92d6175b03f377 (table default.Table_d39b99f0145b444a926eb0ecda4662e6 [id=a6d01f64a2e34c5781f1e090b1d2cd88]) successfully deleted
I20251028 02:59:58.862254 9190 leader_election.cc:290] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:58.862775 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 49b46c13cb4e46388ed0a69d2ced9b42 from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:58.862896 8354 leader_election.cc:343] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: a19eeac727e142579e34a301a6e2db02
I20251028 02:59:58.863436 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4d6d263beb42476ab582fad092f43a53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.863534 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_891be01aa2b540a7b16b407884b6de92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.863912 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d6d263beb42476ab582fad092f43a53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:58.863919 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_891be01aa2b540a7b16b407884b6de92 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:58.864094 8356 leader_election.cc:343] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: a19eeac727e142579e34a301a6e2db02
I20251028 02:59:58.864166 8356 leader_election.cc:304] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:58.864508 9190 raft_consensus.cc:2749] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.864835 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_514bd1c4b2974dc28f976754be3f2c1c" } new_table_name: "default.Table_bfc47b32a50e48e481faae95c5e02679" modify_external_catalogs: true
I20251028 02:59:58.865350 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 38ad9a5b7f0d46d29746d7ed1f8ffda2 (DEFAULT_TABLE table=default.Table_dc0c85038d58432db143e5af2d2f4dab [id=a4323e2b40ba4ddabfd54089fe928c23]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.865499 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 38ad9a5b7f0d46d29746d7ed1f8ffda2 (DEFAULT_TABLE table=default.Table_dc0c85038d58432db143e5af2d2f4dab [id=a4323e2b40ba4ddabfd54089fe928c23]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.865785 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ad9a5b7f0d46d29746d7ed1f8ffda2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.865797 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ad9a5b7f0d46d29746d7ed1f8ffda2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.866683 8545 tablet_bootstrap.cc:492] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.866775 8545 ts_tablet_manager.cc:1403] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.001s
I20251028 02:59:58.867460 8545 raft_consensus.cc:359] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.867561 8545 raft_consensus.cc:385] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.867592 8545 raft_consensus.cc:740] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.867709 8545 consensus_queue.cc:260] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.868011 8545 ts_tablet_manager.cc:1434] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.868211 8545 tablet_bootstrap.cc:492] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.871490 8545 tablet_bootstrap.cc:654] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.874918 9595 raft_consensus.cc:493] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.875147 9595 raft_consensus.cc:515] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.876060 9595 leader_election.cc:290] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.876358 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c21f794ec1284f5d99521954ed7ba9c9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:58.876549 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet b9803f4cfb9b4ef9a1954c83985d6258
W20251028 02:59:58.876761 8220 leader_election.cc:343] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.877077 7964 catalog_manager.cc:6719] ReplaceTablet: tablet b9803f4cfb9b4ef9a1954c83985d6258 deleted and replaced by tablet 2b6f62b942a94af783c026a043cc60a3
I20251028 02:59:58.878149 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet b9803f4cfb9b4ef9a1954c83985d6258 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b6f62b942a94af783c026a043cc60a3) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.878549 8545 tablet_bootstrap.cc:492] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.878640 8545 ts_tablet_manager.cc:1403] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
W20251028 02:59:58.878872 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet b9803f4cfb9b4ef9a1954c83985d6258 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9803f4cfb9b4ef9a1954c83985d6258 already in progress: creating tablet
I20251028 02:59:58.879474 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c21f794ec1284f5d99521954ed7ba9c9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:58.879601 8144 raft_consensus.cc:2468] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:58.879643 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7072983821f0413c95a0ff43b50ada34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.879899 8218 leader_election.cc:304] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
W20251028 02:59:58.880008 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7072983821f0413c95a0ff43b50ada34 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.879210 8545 raft_consensus.cc:359] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.880206 8545 raft_consensus.cc:385] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.880261 8545 raft_consensus.cc:740] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.880472 8545 consensus_queue.cc:260] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.881165 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_621dda8a15654228acd735a8dca754ef" } modify_external_catalogs: true
I20251028 02:59:58.881260 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_621dda8a15654228acd735a8dca754ef" } modify_external_catalogs: true
I20251028 02:59:58.881836 9595 raft_consensus.cc:2804] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:58.882177 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet c03faadb312742a4aecac0325f3595b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.882330 8550 tablet_replica.cc:333] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.882373 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_35c25b81a8284119811feccb77fbf6a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:58.882459 8550 raft_consensus.cc:2243] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.882552 8550 raft_consensus.cc:2272] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:58.882711 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35c25b81a8284119811feccb77fbf6a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.883157 8550 ts_tablet_manager.cc:1916] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.881906 9595 raft_consensus.cc:493] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:58.883366 9595 raft_consensus.cc:3060] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.884596 9595 raft_consensus.cc:515] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.885039 9595 leader_election.cc:290] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:58.885311 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c21f794ec1284f5d99521954ed7ba9c9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:58.885311 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c21f794ec1284f5d99521954ed7ba9c9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:58.885427 8416 raft_consensus.cc:3060] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.885427 8144 raft_consensus.cc:3060] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.885620 8550 ts_tablet_manager.cc:1929] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.885689 8550 log.cc:1199] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c03faadb312742a4aecac0325f3595b7
I20251028 02:59:58.885906 8550 ts_tablet_manager.cc:1950] T c03faadb312742a4aecac0325f3595b7 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.886062 7786 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu with pid 7794
I20251028 02:59:58.886456 8416 raft_consensus.cc:2468] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:58.886592 8144 raft_consensus.cc:2468] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:58.886886 8218 leader_election.cc:304] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:58.887319 8545 ts_tablet_manager.cc:1434] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20251028 02:59:58.887473 8553 raft_consensus.cc:2804] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:58.887562 8553 raft_consensus.cc:697] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:58.887692 8553 consensus_queue.cc:237] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.880566 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2f0fa5294def487f882830d451a7539e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.888649 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2f0fa5294def487f882830d451a7539e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.888676 8545 tablet_bootstrap.cc:492] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.889060 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet c03faadb312742a4aecac0325f3595b7 (table default.Table_7fa5056d9def461dab21615f8b7e8e82 [id=da02d283856342fdaba51b17ce586311]) successfully deleted
I20251028 02:59:58.889830 8545 tablet_bootstrap.cc:654] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.891345 9190 raft_consensus.cc:493] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.891321 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 6455fca4d6684904a255751ce63258f0 (DEFAULT_TABLE table=default.Table_db72e72bf1ce4c70a62bdf4b009366ac [id=1a036b31b861428d909411cd65fdc546]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.891424 9190 raft_consensus.cc:515] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.891587 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6455fca4d6684904a255751ce63258f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.891753 8384 tablet_service.cc:1505] Processing CreateTablet for tablet 6455fca4d6684904a255751ce63258f0 (DEFAULT_TABLE table=default.Table_db72e72bf1ce4c70a62bdf4b009366ac [id=1a036b31b861428d909411cd65fdc546]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.891809 9190 leader_election.cc:290] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:58.892027 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6455fca4d6684904a255751ce63258f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.892062 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 2a57c473fcd74b92819105d83f52b3a2 (DEFAULT_TABLE table=default.Table_ad274b5a83e045a685e2d4322bde36fb [id=b008d7a39fb54a45a6b42872a0bead68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.892146 8251 tablet_service.cc:1505] Processing CreateTablet for tablet e3b35a5a341641738a9e6a719d6baa35 (DEFAULT_TABLE table=default.Table_3355092ddb794533a5545c15b2847d15 [id=3d48f7f5019a449ca4e5d21bd2fca7f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.892258 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a57c473fcd74b92819105d83f52b3a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.892362 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3b35a5a341641738a9e6a719d6baa35. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.893999 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 0cb76fbe3b2446c7bfc1aa23e8be2513 (DEFAULT_TABLE table=default.Table_a54d50989a0842bb8cd3b12aca9bf193 [id=185e21a2b9074adfb0e22c101e698c1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.894248 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cb76fbe3b2446c7bfc1aa23e8be2513. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.894284 8259 tablet_service.cc:1505] Processing CreateTablet for tablet a52b2fe9282b486682a961fc2136557c (DEFAULT_TABLE table=default.Table_e0c668202e06476cbbf5c2d3ecda31b3 [id=d2a2c394e6434445abdeb95f4554ab4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.894503 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a52b2fe9282b486682a961fc2136557c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.894553 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa0776d1a8ef43449309221b86ed3923" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.894660 8271 raft_consensus.cc:2468] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:58.894943 8356 leader_election.cc:304] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:58.895109 9190 raft_consensus.cc:2804] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:58.895151 9190 raft_consensus.cc:493] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:58.895185 9190 raft_consensus.cc:3060] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.896199 9190 raft_consensus.cc:515] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.896608 9190 leader_election.cc:290] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:58.896726 8387 tablet_service.cc:1505] Processing CreateTablet for tablet a52b2fe9282b486682a961fc2136557c (DEFAULT_TABLE table=default.Table_e0c668202e06476cbbf5c2d3ecda31b3 [id=d2a2c394e6434445abdeb95f4554ab4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.896981 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a52b2fe9282b486682a961fc2136557c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.897220 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa0776d1a8ef43449309221b86ed3923" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:58.897321 8271 raft_consensus.cc:3060] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.897297 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 0cb76fbe3b2446c7bfc1aa23e8be2513 (DEFAULT_TABLE table=default.Table_a54d50989a0842bb8cd3b12aca9bf193 [id=185e21a2b9074adfb0e22c101e698c1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.897495 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cb76fbe3b2446c7bfc1aa23e8be2513. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.898185 8271 raft_consensus.cc:2468] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:58.898437 8356 leader_election.cc:304] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
W20251028 02:59:58.898490 8354 leader_election.cc:343] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: aa0776d1a8ef43449309221b86ed3923
I20251028 02:59:58.898653 9190 raft_consensus.cc:2804] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:58.898746 9190 raft_consensus.cc:697] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:58.898876 9190 consensus_queue.cc:237] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 02:59:58.899164 8354 leader_election.cc:343] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: aa0776d1a8ef43449309221b86ed3923
I20251028 02:59:58.900046 8387 tablet_service.cc:1505] Processing CreateTablet for tablet e3b35a5a341641738a9e6a719d6baa35 (DEFAULT_TABLE table=default.Table_3355092ddb794533a5545c15b2847d15 [id=3d48f7f5019a449ca4e5d21bd2fca7f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.900084 8384 tablet_service.cc:1505] Processing CreateTablet for tablet 2a57c473fcd74b92819105d83f52b3a2 (DEFAULT_TABLE table=default.Table_ad274b5a83e045a685e2d4322bde36fb [id=b008d7a39fb54a45a6b42872a0bead68]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.900700 8544 tablet_bootstrap.cc:492] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.900872 8544 ts_tablet_manager.cc:1403] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.055s user 0.003s sys 0.000s
I20251028 02:59:58.901816 8544 raft_consensus.cc:359] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.902009 8544 raft_consensus.cc:385] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.902088 8544 raft_consensus.cc:740] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.902362 8544 consensus_queue.cc:260] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.902745 8544 ts_tablet_manager.cc:1434] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:58.903321 8544 tablet_bootstrap.cc:492] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.903739 8398 tablet_service.cc:1505] Processing CreateTablet for tablet d52cd7ecd9c741d2a59e06396b5b384a (DEFAULT_TABLE table=default.Table_5d43bd697e934ca4a3b0e56386389091 [id=98bbb904e86a4ea28a1cccaf618d62c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.904457 8544 tablet_bootstrap.cc:654] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.904579 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 848dbb23d5604a7085947771c5ee3d5e (DEFAULT_TABLE table=default.Table_f2dee7552f5543b0ac6899b7a28431a2 [id=715f093c061941709983b951959cb8c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.904850 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 848dbb23d5604a7085947771c5ee3d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.905529 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 848dbb23d5604a7085947771c5ee3d5e (DEFAULT_TABLE table=default.Table_f2dee7552f5543b0ac6899b7a28431a2 [id=715f093c061941709983b951959cb8c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.906141 8256 tablet_service.cc:1505] Processing CreateTablet for tablet d52cd7ecd9c741d2a59e06396b5b384a (DEFAULT_TABLE table=default.Table_5d43bd697e934ca4a3b0e56386389091 [id=98bbb904e86a4ea28a1cccaf618d62c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.906375 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d52cd7ecd9c741d2a59e06396b5b384a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.906427 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 46616e3a5c934d698a097fa1565a4a08 (DEFAULT_TABLE table=default.Table_129667a395814d7fa62be13abf2feed1 [id=cc97298f5e594d2487b31cfb0af17186]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.906620 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46616e3a5c934d698a097fa1565a4a08. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.906967 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 46616e3a5c934d698a097fa1565a4a08 (DEFAULT_TABLE table=default.Table_129667a395814d7fa62be13abf2feed1 [id=cc97298f5e594d2487b31cfb0af17186]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.907541 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 680f8235c6e64a48bd0009c699760691 (DEFAULT_TABLE table=default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.907740 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 680f8235c6e64a48bd0009c699760691. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.910058 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3b35a5a341641738a9e6a719d6baa35. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:58.910655 8218 consensus_peers.cc:597] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda -> Peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Couldn't send request to peer fbd2afb4894c4e7a8520cc09f8593aa0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b92443ae55f4a489d92d6175b03f377. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:58.910876 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d52cd7ecd9c741d2a59e06396b5b384a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.911338 8546 tablet_bootstrap.cc:492] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.911422 8546 ts_tablet_manager.cc:1403] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.067s user 0.002s sys 0.000s
I20251028 02:59:58.911841 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 848dbb23d5604a7085947771c5ee3d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.911926 8546 raft_consensus.cc:359] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.912076 8546 raft_consensus.cc:385] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.912119 8546 raft_consensus.cc:740] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.912237 8546 consensus_queue.cc:260] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.912259 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet c21f794ec1284f5d99521954ed7ba9c9 (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]). Expected version 1 got 0
I20251028 02:59:58.912376 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46616e3a5c934d698a097fa1565a4a08. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.912554 8546 ts_tablet_manager.cc:1434] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.912782 8546 tablet_bootstrap.cc:492] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:58.912789 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99cc3e2b1eef4137bac27c629600f43b consensus state has no leader
I20251028 02:59:58.913851 8546 tablet_bootstrap.cc:654] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.914135 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b764dce6b08a49c5a196bd97d3297727"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.914531 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b764dce6b08a49c5a196bd97d3297727 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.914563 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet c03faadb312742a4aecac0325f3595b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.914721 7965 catalog_manager.cc:5649] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:58.914959 7965 catalog_manager.cc:5649] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:58.915089 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet c21f794ec1284f5d99521954ed7ba9c9 (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]). Expected version 1 got 0
W20251028 02:59:58.915354 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ad374b7d30414afa91eb06e439a7081b consensus state has no leader
I20251028 02:59:58.915551 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_acbe587ace4042f29f736a17a6d2530a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.915874 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_acbe587ace4042f29f736a17a6d2530a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.916785 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a57c473fcd74b92819105d83f52b3a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.918779 8551 ts_tablet_manager.cc:1929] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.918856 8551 log.cc:1199] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/f6ac7b5fff0848cebf710ba3ed50def4
I20251028 02:59:58.919070 8551 ts_tablet_manager.cc:1950] T f6ac7b5fff0848cebf710ba3ed50def4 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.919644 8256 tablet_service.cc:1505] Processing CreateTablet for tablet af23fe1145a54b76a9f8e7249b87ca0d (DEFAULT_TABLE table=default.Table_18181568bf2c4744a84690b6dac5f13f [id=eb8a8d39e6f84bde9ed20d3c510baa24]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.919936 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af23fe1145a54b76a9f8e7249b87ca0d. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:58.919939 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet c03faadb312742a4aecac0325f3595b7 because the tablet was not found. No further retry: Not found: Tablet not found: c03faadb312742a4aecac0325f3595b7
I20251028 02:59:58.920245 8546 tablet_bootstrap.cc:492] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.920307 8546 ts_tablet_manager.cc:1403] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 02:59:58.920706 8546 raft_consensus.cc:359] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.920821 8546 raft_consensus.cc:385] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.920856 8546 raft_consensus.cc:740] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.920972 8546 consensus_queue.cc:260] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.921254 8546 ts_tablet_manager.cc:1434] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:58.921428 8546 tablet_bootstrap.cc:492] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.922461 8546 tablet_bootstrap.cc:654] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.922850 8396 tablet_service.cc:1505] Processing CreateTablet for tablet af23fe1145a54b76a9f8e7249b87ca0d (DEFAULT_TABLE table=default.Table_18181568bf2c4744a84690b6dac5f13f [id=eb8a8d39e6f84bde9ed20d3c510baa24]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.923105 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af23fe1145a54b76a9f8e7249b87ca0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.923202 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 49b46c13cb4e46388ed0a69d2ced9b42
I20251028 02:59:58.923520 7967 catalog_manager.cc:6719] ReplaceTablet: tablet 49b46c13cb4e46388ed0a69d2ced9b42 deleted and replaced by tablet 1f57930d5211413cb6cf4a693f7557d1
I20251028 02:59:58.923707 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3d3d9ba2e032453fbbc6b2635d1b2203" } new_table_name: "default.Table_24ec851d131d4496955e5cb7f05d562e" modify_external_catalogs: true
I20251028 02:59:58.924154 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet b46554b0df894beb8141126aa7ee7b8e (table default.Table_176b16f7b1fd4a30991b82efd4ddd497 [id=9e5ad11a9ea34a3a8c629f48142b68f0]). Expected version 1 got 0
I20251028 02:59:58.924628 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 0868c69e28a840b4ab9dad61d50983ec from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.924953 9190 raft_consensus.cc:493] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.925030 9190 raft_consensus.cc:515] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.925388 9190 leader_election.cc:290] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:58.925488 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 49b46c13cb4e46388ed0a69d2ced9b42 with delete_type TABLET_DATA_DELETED (replaced by tablet 1f57930d5211413cb6cf4a693f7557d1) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.925818 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "920623f3747046d9be594241e8c13733" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:58.926153 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 746eae30ca284758a6e3ed8ad80c2cdc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.926453 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 746eae30ca284758a6e3ed8ad80c2cdc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.926563 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "920623f3747046d9be594241e8c13733" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:58.926707 8145 raft_consensus.cc:2393] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7e4cf694f71438298dd32a346abcf75 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:58.926977 8550 tablet_replica.cc:333] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.927140 8550 raft_consensus.cc:2243] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.927179 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 12d103e9b5024a87aa232c26e0c51ab9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.927261 8550 raft_consensus.cc:2272] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.927668 8550 ts_tablet_manager.cc:1916] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.927733 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 56c8969da9dc464d9b463205e3a27965 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:58.927855 8354 leader_election.cc:304] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:58.928299 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 12d103e9b5024a87aa232c26e0c51ab9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.928596 9190 raft_consensus.cc:3060] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:58.928987 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b0c593a3b96434db9ac42d260fc6997 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.929425 9190 raft_consensus.cc:2749] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:58.929592 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b0c593a3b96434db9ac42d260fc6997 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.929888 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet e369dbabc2c04ad5ab17bbd4225127c1
I20251028 02:59:58.930159 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b43da9580244346ae512a7bc9b506d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.930244 8550 ts_tablet_manager.cc:1929] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:58.930306 8550 log.cc:1199] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/49b46c13cb4e46388ed0a69d2ced9b42
I20251028 02:59:58.930297 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5a485f2c39204b9b8ed300bb7e924bed" } new_table_name: "default.Table_1e52c58d3e4042139159ba0e312a4089" modify_external_catalogs: true
I20251028 02:59:58.930401 8552 tablet_replica.cc:333] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.930553 8552 raft_consensus.cc:2243] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.930667 8552 raft_consensus.cc:2272] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.930920 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet e369dbabc2c04ad5ab17bbd4225127c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.931087 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet d60fc654f6af44b290d46235082a4113 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d42a5987fa540ce86afdbbe1922896e) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.931399 8552 ts_tablet_manager.cc:1916] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.931458 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_096243ad06e04bfba7e8b97d066cdf9c" } new_table_name: "default.Table_e95d99935d7d4d94a16f5a826847ce32" modify_external_catalogs: true
I20251028 02:59:58.932462 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e29ed40d642b400192ea2dc90e9c28b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:58.932830 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e29ed40d642b400192ea2dc90e9c28b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:58.933768 8552 ts_tablet_manager.cc:1929] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.933836 8552 log.cc:1199] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3b43da9580244346ae512a7bc9b506d3
I20251028 02:59:58.934001 8546 tablet_bootstrap.cc:492] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.934056 8552 ts_tablet_manager.cc:1950] T 3b43da9580244346ae512a7bc9b506d3 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.934096 8546 ts_tablet_manager.cc:1403] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20251028 02:59:58.934592 8546 raft_consensus.cc:359] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.934669 8546 raft_consensus.cc:385] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.934691 8546 raft_consensus.cc:740] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.934768 8546 consensus_queue.cc:260] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.935019 8546 ts_tablet_manager.cc:1434] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.935153 8546 tablet_bootstrap.cc:492] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.936414 8546 tablet_bootstrap.cc:654] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.937563 9551 tablet.cc:1722] T b46554b0df894beb8141126aa7ee7b8e P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:58.937896 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet c21f794ec1284f5d99521954ed7ba9c9 (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]). Expected version 1 got 0
I20251028 02:59:58.930943 8550 ts_tablet_manager.cc:1950] T 49b46c13cb4e46388ed0a69d2ced9b42 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.938477 8384 tablet_service.cc:1505] Processing CreateTablet for tablet 680f8235c6e64a48bd0009c699760691 (DEFAULT_TABLE table=default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.938766 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 680f8235c6e64a48bd0009c699760691. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.940162 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 49b46c13cb4e46388ed0a69d2ced9b42 (table default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]) successfully deleted
W20251028 02:59:58.940318 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e369dbabc2c04ad5ab17bbd4225127c1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e369dbabc2c04ad5ab17bbd4225127c1 already in progress: creating tablet
I20251028 02:59:58.940562 9529 tablet.cc:1722] T b46554b0df894beb8141126aa7ee7b8e P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251028 02:59:58.941170 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2bf100407b034626b4a9f7e2340f9cf0 consensus state has no leader
I20251028 02:59:58.941790 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 56c8969da9dc464d9b463205e3a27965
I20251028 02:59:58.942061 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 56c8969da9dc464d9b463205e3a27965 deleted and replaced by tablet 4a57e122bbb94a9fac8c8254ab73a21f
I20251028 02:59:58.942413 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 56c8969da9dc464d9b463205e3a27965 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a57e122bbb94a9fac8c8254ab73a21f) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.941781 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 0868c69e28a840b4ab9dad61d50983ec
I20251028 02:59:58.943043 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 0868c69e28a840b4ab9dad61d50983ec with delete_type TABLET_DATA_DELETED (replaced by tablet 88cb16241e364f56b251d88079936c7b) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.943280 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 900973d832c24b62a4554ae9a219cfd8 (table default.Table_f0dafbe2e0a84541a79cf6c887d59619 [id=c4aadbebe43f451ea22467feaa6dafbb]). Expected version 1 got 0
I20251028 02:59:58.943499 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet c21f794ec1284f5d99521954ed7ba9c9 (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]). Expected version 1 got 0
I20251028 02:59:58.943747 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 0d42a5987fa540ce86afdbbe1922896e (DEFAULT_TABLE table=default.Table_a387569232e54415a0dd77e906ddcee4 [id=94895dadf0e34a19a454e8755c61c785]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.944007 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d42a5987fa540ce86afdbbe1922896e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.944280 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f06af61617e4be9a753692e9ad4978d with delete_type TABLET_DATA_DELETED (replaced by tablet 26a60624333d43f1b701036b7e9dfdb0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.944455 8551 tablet_replica.cc:333] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.944597 8551 raft_consensus.cc:2243] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.944710 8551 raft_consensus.cc:2272] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.945621 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet d60fc654f6af44b290d46235082a4113 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d42a5987fa540ce86afdbbe1922896e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.945659 8551 ts_tablet_manager.cc:1916] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.946650 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 0868c69e28a840b4ab9dad61d50983ec deleted and replaced by tablet 88cb16241e364f56b251d88079936c7b
I20251028 02:59:58.946935 8550 tablet_replica.cc:333] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.947198 8550 raft_consensus.cc:2243] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:58.947705 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 0d42a5987fa540ce86afdbbe1922896e (DEFAULT_TABLE table=default.Table_a387569232e54415a0dd77e906ddcee4 [id=94895dadf0e34a19a454e8755c61c785]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.947818 8550 raft_consensus.cc:2272] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.947944 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b92443ae55f4a489d92d6175b03f377 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.947979 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d42a5987fa540ce86afdbbe1922896e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.948374 8550 ts_tablet_manager.cc:1916] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:58.949159 8221 connection.cc:537] client connection to 127.7.154.190:39091 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251028 02:59:58.950057 7956 connection.cc:537] client connection to 127.7.154.190:39091 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251028 02:59:58.950125 7956 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 5 similar messages]
W20251028 02:59:58.950992 8084 connection.cc:537] client connection to 127.7.154.190:39091 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251028 02:59:58.951179 8550 ts_tablet_manager.cc:1929] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:58.951242 8550 log.cc:1199] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/56c8969da9dc464d9b463205e3a27965
I20251028 02:59:58.951545 8550 ts_tablet_manager.cc:1950] T 56c8969da9dc464d9b463205e3a27965 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.952250 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 56c8969da9dc464d9b463205e3a27965 (table default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]) successfully deleted
I20251028 02:59:58.952544 8550 tablet_replica.cc:333] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.952718 8550 raft_consensus.cc:2243] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.952909 8550 raft_consensus.cc:2272] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.953366 8550 ts_tablet_manager.cc:1916] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.955576 8550 ts_tablet_manager.cc:1929] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:58.955714 8550 log.cc:1199] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/0868c69e28a840b4ab9dad61d50983ec
I20251028 02:59:58.956038 8550 ts_tablet_manager.cc:1950] T 0868c69e28a840b4ab9dad61d50983ec P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.957059 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 0868c69e28a840b4ab9dad61d50983ec (table default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]) successfully deleted
I20251028 02:59:58.957475 8545 tablet_bootstrap.cc:492] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.957580 8545 ts_tablet_manager.cc:1403] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.069s user 0.002s sys 0.000s
I20251028 02:59:58.958114 8545 raft_consensus.cc:359] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.958226 8545 raft_consensus.cc:385] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.958261 8545 raft_consensus.cc:740] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.958379 8545 consensus_queue.cc:260] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.958690 8545 ts_tablet_manager.cc:1434] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.958909 8545 tablet_bootstrap.cc:492] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 02:59:58.959860 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 746eae30ca284758a6e3ed8ad80c2cdc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 746eae30ca284758a6e3ed8ad80c2cdc already in progress: creating tablet
I20251028 02:59:58.959967 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_176b16f7b1fd4a30991b82efd4ddd497 [id=9e5ad11a9ea34a3a8c629f48142b68f0] alter complete (version 1)
I20251028 02:59:58.959995 8545 tablet_bootstrap.cc:654] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:58.960009 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 12d103e9b5024a87aa232c26e0c51ab9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12d103e9b5024a87aa232c26e0c51ab9 already in progress: creating tablet
W20251028 02:59:58.960031 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 583d6226a5f44d47a5ea4788c2b0d5ce consensus state has no leader
W20251028 02:59:58.960115 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 3b0c593a3b96434db9ac42d260fc6997 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b0c593a3b96434db9ac42d260fc6997 already in progress: creating tablet
I20251028 02:59:58.960220 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3b43da9580244346ae512a7bc9b506d3 (table default.Table_a00976e88f0147c5b01560dba597ea82 [id=7244d96a49d34411bb8f9f7416904ae3]) successfully deleted
W20251028 02:59:58.960410 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet d60fc654f6af44b290d46235082a4113 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d60fc654f6af44b290d46235082a4113 already in progress: creating tablet
I20251028 02:59:58.960657 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 74120decd53d4de19c5f6fedf70ecaae (table default.Table_5002cbe7bf3044e4baaa8e884eb9283f [id=5290176e2166464799b4d2f4d54e97ff]) successfully deleted
W20251028 02:59:58.961496 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 464e6b3483bc4816bfa844c52fd3ca53 consensus state has no leader
I20251028 02:59:58.962226 8551 ts_tablet_manager.cc:1929] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.962291 8551 log.cc:1199] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3f06af61617e4be9a753692e9ad4978d
I20251028 02:59:58.962510 8551 ts_tablet_manager.cc:1950] T 3f06af61617e4be9a753692e9ad4978d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.962589 9685 raft_consensus.cc:493] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:58.962677 9685 raft_consensus.cc:515] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.963078 9685 leader_election.cc:290] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:58.963809 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b92443ae55f4a489d92d6175b03f377 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:58.963889 8218 leader_election.cc:343] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 83059074c365440b84e39cf7a5553eb2
I20251028 02:59:58.963878 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "83059074c365440b84e39cf7a5553eb2" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:58.960790 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 06f7d43bea27429fb45413e0dc0ba269 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06f7d43bea27429fb45413e0dc0ba269 already in progress: creating tablet
I20251028 02:59:58.964040 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet f08a6e30211a4d59934c36c7a26ac983 (table default.Table_c62970e81d90490ebe817e6ff5c68b97 [id=b9c53ab5608f49309950ed79b8272710]) successfully deleted
I20251028 02:59:58.964154 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet d59acfb0e12643ebbb406dbda7bef29b (table default.Table_e4f1369ad6ad4110b5854544d4a266bd [id=ccea14281b2c420a99cede953a8de44d]) successfully deleted
I20251028 02:59:58.964160 8552 tablet_replica.cc:333] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.964325 8552 raft_consensus.cc:2243] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
W20251028 02:59:58.964370 8220 leader_election.cc:343] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:58.964409 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet f6ac7b5fff0848cebf710ba3ed50def4 (table default.Table_42f89d7fccbf4c738293e9796be235a0 [id=6f3e689a22ae4160ae7537d955c06a90]) successfully deleted
I20251028 02:59:58.964425 8220 leader_election.cc:304] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:58.964545 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 746eae30ca284758a6e3ed8ad80c2cdc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 746eae30ca284758a6e3ed8ad80c2cdc already in progress: creating tablet
I20251028 02:59:58.964612 9685 raft_consensus.cc:2749] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251028 02:59:58.964646 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 12d103e9b5024a87aa232c26e0c51ab9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12d103e9b5024a87aa232c26e0c51ab9 already in progress: creating tablet
W20251028 02:59:58.964740 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 3b0c593a3b96434db9ac42d260fc6997 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b0c593a3b96434db9ac42d260fc6997 already in progress: creating tablet
I20251028 02:59:58.964790 8552 raft_consensus.cc:2272] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:58.964861 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet d60fc654f6af44b290d46235082a4113 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d60fc654f6af44b290d46235082a4113 already in progress: creating tablet
I20251028 02:59:58.965227 8545 tablet_bootstrap.cc:492] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.965250 8552 ts_tablet_manager.cc:1916] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.965325 8545 ts_tablet_manager.cc:1403] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 02:59:58.965590 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet b9803f4cfb9b4ef9a1954c83985d6258 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b6f62b942a94af783c026a043cc60a3) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.965847 8545 raft_consensus.cc:359] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.965960 8545 raft_consensus.cc:385] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.966058 8545 raft_consensus.cc:740] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.966212 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet 45572222af854b34a5f26a00f8ad9615 with delete_type TABLET_DATA_DELETED (replaced by tablet d60fc654f6af44b290d46235082a4113) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.966231 8545 consensus_queue.cc:260] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:58.966657 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet a19eeac727e142579e34a301a6e2db02 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.967577 8552 ts_tablet_manager.cc:1929] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:58.967609 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet aa0776d1a8ef43449309221b86ed3923 with delete_type TABLET_DATA_DELETED (replaced by tablet ad926e9ee0044c62a7abbfca959a4f56) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.967770 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 49b46c13cb4e46388ed0a69d2ced9b42 with delete_type TABLET_DATA_DELETED (replaced by tablet 1f57930d5211413cb6cf4a693f7557d1) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.967998 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet e369dbabc2c04ad5ab17bbd4225127c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.969455 8552 log.cc:1199] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9b92443ae55f4a489d92d6175b03f377
I20251028 02:59:58.968407 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 49b46c13cb4e46388ed0a69d2ced9b42 with delete_type TABLET_DATA_DELETED (replaced by tablet 1f57930d5211413cb6cf4a693f7557d1) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.968210 8545 ts_tablet_manager.cc:1434] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:58.968528 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet e369dbabc2c04ad5ab17bbd4225127c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.968295 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet b9803f4cfb9b4ef9a1954c83985d6258 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b6f62b942a94af783c026a043cc60a3) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.969875 8545 tablet_bootstrap.cc:492] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.969578 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet aa0776d1a8ef43449309221b86ed3923 with delete_type TABLET_DATA_DELETED (replaced by tablet ad926e9ee0044c62a7abbfca959a4f56) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.970011 8552 ts_tablet_manager.cc:1950] T 9b92443ae55f4a489d92d6175b03f377 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:58.969575 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 56c8969da9dc464d9b463205e3a27965 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a57e122bbb94a9fac8c8254ab73a21f) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.969385 8551 tablet_replica.cc:333] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.970431 8551 raft_consensus.cc:2243] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.970633 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 0868c69e28a840b4ab9dad61d50983ec with delete_type TABLET_DATA_DELETED (replaced by tablet 88cb16241e364f56b251d88079936c7b) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.970783 8551 raft_consensus.cc:2272] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.971213 8551 ts_tablet_manager.cc:1916] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.972925 8545 tablet_bootstrap.cc:654] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.973419 8551 ts_tablet_manager.cc:1929] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:58.973489 8551 log.cc:1199] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9b92443ae55f4a489d92d6175b03f377
I20251028 02:59:58.973731 8551 ts_tablet_manager.cc:1950] T 9b92443ae55f4a489d92d6175b03f377 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:58.974336 8551 tablet_replica.cc:333] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:58.974462 8551 raft_consensus.cc:2243] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.974540 8551 raft_consensus.cc:2272] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.970142 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 56c8969da9dc464d9b463205e3a27965 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a57e122bbb94a9fac8c8254ab73a21f) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.975323 8552 tablet_replica.cc:333] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:58.975505 8552 raft_consensus.cc:2243] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:58.975760 8552 raft_consensus.cc:2272] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:58.975926 9821 negotiation.cc:337] Failed RPC negotiation. Trace:
1028 02:59:58.949667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.154.190:39091 (local address 127.7.154.130:53029)
1028 02:59:58.975713 (+ 26046us) negotiation.cc:107] Waiting for socket to connect
1028 02:59:58.975788 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.154.190:39091: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":25951,"thread_start_us":119,"threads_started":1}
I20251028 02:59:58.970856 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 0868c69e28a840b4ab9dad61d50983ec with delete_type TABLET_DATA_DELETED (replaced by tablet 88cb16241e364f56b251d88079936c7b) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:58.976352 9820 negotiation.cc:337] Failed RPC negotiation. Trace:
1028 02:59:58.948895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.154.190:39091 (local address 127.7.154.131:47417)
1028 02:59:58.976198 (+ 27303us) negotiation.cc:107] Waiting for socket to connect
1028 02:59:58.976259 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.154.190:39091: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":27221,"thread_start_us":73,"threads_started":1}
I20251028 02:59:58.976503 8552 ts_tablet_manager.cc:1916] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.978436 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9b92443ae55f4a489d92d6175b03f377 (table default.Table_d39b99f0145b444a926eb0ecda4662e6 [id=a6d01f64a2e34c5781f1e090b1d2cd88]) successfully deleted
I20251028 02:59:58.978608 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3f06af61617e4be9a753692e9ad4978d (table default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3]) successfully deleted
I20251028 02:59:58.978657 8544 tablet_bootstrap.cc:492] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:58.978710 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9b92443ae55f4a489d92d6175b03f377 (table default.Table_d39b99f0145b444a926eb0ecda4662e6 [id=a6d01f64a2e34c5781f1e090b1d2cd88]) successfully deleted
I20251028 02:59:58.978739 8544 ts_tablet_manager.cc:1403] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.075s user 0.002s sys 0.000s
W20251028 02:59:58.978856 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet b9803f4cfb9b4ef9a1954c83985d6258 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9803f4cfb9b4ef9a1954c83985d6258 already in progress: creating tablet
I20251028 02:59:58.979215 8544 raft_consensus.cc:359] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.979341 8544 raft_consensus.cc:385] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.979372 8544 raft_consensus.cc:740] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.979488 8544 consensus_queue.cc:260] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:58.979738 8544 ts_tablet_manager.cc:1434] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:58.979935 8544 tablet_bootstrap.cc:492] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:58.981009 8544 tablet_bootstrap.cc:654] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.981079 8546 tablet_bootstrap.cc:492] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:58.981061 8113 tablet_service.cc:1505] Processing CreateTablet for tablet a67c21155a494d2cab8af8ca6668c1d8 (DEFAULT_TABLE table=default.Table_1f11d33cf9ac4f11bc540fa516f55d2b [id=ec107c12db6243f4987558b19e1c4a8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.981171 8546 ts_tablet_manager.cc:1403] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.000s
I20251028 02:59:58.981318 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67c21155a494d2cab8af8ca6668c1d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.981751 8546 raft_consensus.cc:359] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.981863 8546 raft_consensus.cc:385] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.981897 8546 raft_consensus.cc:740] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.982033 8546 consensus_queue.cc:260] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.982306 8546 ts_tablet_manager.cc:1434] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:58.982497 8546 tablet_bootstrap.cc:492] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:58.983510 8546 tablet_bootstrap.cc:654] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.984056 8400 tablet_service.cc:1505] Processing CreateTablet for tablet a67c21155a494d2cab8af8ca6668c1d8 (DEFAULT_TABLE table=default.Table_1f11d33cf9ac4f11bc540fa516f55d2b [id=ec107c12db6243f4987558b19e1c4a8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.984328 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67c21155a494d2cab8af8ca6668c1d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.989830 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f04c867f0ce4ab99f980cca063abc30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:58.990116 8550 tablet_replica.cc:333] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:58.990165 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b43da9580244346ae512a7bc9b506d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:58.990245 8550 raft_consensus.cc:2243] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:58.990350 8550 raft_consensus.cc:2272] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:58.990907 8550 ts_tablet_manager.cc:1916] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:58.991151 7956 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d -> Peer ac9bf5a198d042698c9f030a59228865 (127.7.154.190:39091): Couldn't send request to peer ac9bf5a198d042698c9f030a59228865. Status: Network error: Client connection negotiation failed: client connection to 127.7.154.190:39091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:58.992408 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 73a45776f8964e68b79bc973097223a3 (table default.Table_d246152b26344aed845d6366d95b40a0 [id=a9b54811e6ba4461a372d76424f07b4f]). Expected version 2 got 0
I20251028 02:59:58.993196 8545 tablet_bootstrap.cc:492] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:58.993314 8545 ts_tablet_manager.cc:1403] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20251028 02:59:58.993590 8259 tablet_service.cc:1505] Processing CreateTablet for tablet a67c21155a494d2cab8af8ca6668c1d8 (DEFAULT_TABLE table=default.Table_1f11d33cf9ac4f11bc540fa516f55d2b [id=ec107c12db6243f4987558b19e1c4a8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:58.993818 8551 ts_tablet_manager.cc:1916] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:58.993891 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67c21155a494d2cab8af8ca6668c1d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:58.993881 8545 raft_consensus.cc:359] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.993999 8545 raft_consensus.cc:385] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:58.994050 8545 raft_consensus.cc:740] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:58.994539 8545 consensus_queue.cc:260] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:58.994900 8545 ts_tablet_manager.cc:1434] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:58.995260 8545 tablet_bootstrap.cc:492] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:58.995391 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 83059074c365440b84e39cf7a5553eb2 with delete_type TABLET_DATA_DELETED (replaced by tablet c120a3d56f6b41c28c7f7dcd84359a60) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:58.996414 8545 tablet_bootstrap.cc:654] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:58.997473 8550 ts_tablet_manager.cc:1929] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:58.997551 8550 log.cc:1199] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3f04c867f0ce4ab99f980cca063abc30
I20251028 02:59:58.997785 8550 ts_tablet_manager.cc:1950] T 3f04c867f0ce4ab99f980cca063abc30 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:58.998451 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3f04c867f0ce4ab99f980cca063abc30 (table default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]) successfully deleted
I20251028 02:59:59.000963 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f04c867f0ce4ab99f980cca063abc30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:59.001441 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 3f04c867f0ce4ab99f980cca063abc30 because the tablet was not found. No further retry: Not found: Tablet not found: 3f04c867f0ce4ab99f980cca063abc30
I20251028 02:59:59.001978 8544 tablet_bootstrap.cc:492] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.002080 8544 ts_tablet_manager.cc:1403] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20251028 02:59:59.002513 8544 raft_consensus.cc:359] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.002616 8544 raft_consensus.cc:385] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.002652 8544 raft_consensus.cc:740] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.002894 8544 consensus_queue.cc:260] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.003211 8544 ts_tablet_manager.cc:1434] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.004936 8552 ts_tablet_manager.cc:1929] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.006387 8552 log.cc:1199] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/49b46c13cb4e46388ed0a69d2ced9b42
I20251028 02:59:59.006738 8552 ts_tablet_manager.cc:1950] T 49b46c13cb4e46388ed0a69d2ced9b42 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.007503 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 49b46c13cb4e46388ed0a69d2ced9b42 (table default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]) successfully deleted
I20251028 02:59:59.008224 8551 ts_tablet_manager.cc:1929] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.008307 8551 log.cc:1199] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/45572222af854b34a5f26a00f8ad9615
I20251028 02:59:59.008406 8544 tablet_bootstrap.cc:492] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.008545 8551 ts_tablet_manager.cc:1950] T 45572222af854b34a5f26a00f8ad9615 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.009160 8551 tablet_replica.cc:333] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.009258 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 45572222af854b34a5f26a00f8ad9615 (table default.Table_a387569232e54415a0dd77e906ddcee4 [id=94895dadf0e34a19a454e8755c61c785]) successfully deleted
I20251028 02:59:59.009316 8551 raft_consensus.cc:2243] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.009420 8551 raft_consensus.cc:2272] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.010130 8544 tablet_bootstrap.cc:654] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:59.010362 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet e369dbabc2c04ad5ab17bbd4225127c1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e369dbabc2c04ad5ab17bbd4225127c1 already in progress: creating tablet
I20251028 02:59:59.010568 8552 tablet_replica.cc:333] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 02:59:59.010813 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet b9803f4cfb9b4ef9a1954c83985d6258 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9803f4cfb9b4ef9a1954c83985d6258 already in progress: creating tablet
I20251028 02:59:59.011014 8552 raft_consensus.cc:2243] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.011140 8552 raft_consensus.cc:2272] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.011575 8552 ts_tablet_manager.cc:1916] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.013594 8552 ts_tablet_manager.cc:1929] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.013664 8552 log.cc:1199] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/aa0776d1a8ef43449309221b86ed3923
I20251028 02:59:59.014386 8551 ts_tablet_manager.cc:1916] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.016574 8551 ts_tablet_manager.cc:1929] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.016651 8551 log.cc:1199] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a19eeac727e142579e34a301a6e2db02
I20251028 02:59:59.017015 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 23790dc59186409b9cc9f4d6cd2d2e4e (DEFAULT_TABLE table=default.Table_b764dce6b08a49c5a196bd97d3297727 [id=b5d105fb03d04078a9e25db280a32c45]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.017261 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23790dc59186409b9cc9f4d6cd2d2e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.017558 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 23790dc59186409b9cc9f4d6cd2d2e4e (DEFAULT_TABLE table=default.Table_b764dce6b08a49c5a196bd97d3297727 [id=b5d105fb03d04078a9e25db280a32c45]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.017886 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23790dc59186409b9cc9f4d6cd2d2e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.014282 8546 tablet_bootstrap.cc:492] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.019115 8546 ts_tablet_manager.cc:1403] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.000s
I20251028 02:59:59.019551 8551 ts_tablet_manager.cc:1950] T a19eeac727e142579e34a301a6e2db02 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.019852 8545 tablet_bootstrap.cc:492] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.019791 8546 raft_consensus.cc:359] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.019909 8546 raft_consensus.cc:385] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.019953 8546 raft_consensus.cc:740] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.020066 8545 ts_tablet_manager.cc:1403] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.002s
I20251028 02:59:59.020083 8546 consensus_queue.cc:260] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.020380 8546 ts_tablet_manager.cc:1434] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.020397 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a19eeac727e142579e34a301a6e2db02 (table default.Table_0ac514a2be6a4e1b904149fd8fdc8eb3 [id=effb15303ea6421a8eebaa32a124678e]) successfully deleted
I20251028 02:59:59.020608 8546 tablet_bootstrap.cc:492] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.020553 8545 raft_consensus.cc:359] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.020661 8545 raft_consensus.cc:385] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.020699 8545 raft_consensus.cc:740] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.020846 8545 consensus_queue.cc:260] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.021693 8546 tablet_bootstrap.cc:654] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.022271 8113 tablet_service.cc:1505] Processing CreateTablet for tablet f92acda752d5463aae30ce7e63f2f16f (DEFAULT_TABLE table=default.Table_acbe587ace4042f29f736a17a6d2530a [id=c5de3c9bf3524daf845f65158d3b14b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.022557 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92acda752d5463aae30ce7e63f2f16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.022665 8545 ts_tablet_manager.cc:1434] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251028 02:59:59.022852 8545 tablet_bootstrap.cc:492] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.023178 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 1de3acf65f744368ab31dc1c82ac554f (table default.Table_e5fa1e8f3825450685059a8a45433ef8 [id=739f0673774d472bbb62f176b70998bc]). Expected version 2 got 0
I20251028 02:59:59.023836 8551 tablet_replica.cc:333] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.023954 8551 raft_consensus.cc:2243] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:59.024191 8551 pending_rounds.cc:70] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:59.024225 8551 pending_rounds.cc:77] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597153891082240 op_type: NO_OP noop_request { }
I20251028 02:59:59.024281 8551 raft_consensus.cc:2889] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:59.024331 8551 raft_consensus.cc:2272] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.025099 8545 tablet_bootstrap.cc:654] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.026351 8388 tablet_service.cc:1505] Processing CreateTablet for tablet e0576bc52f99440b9a6e7caf92309bec (DEFAULT_TABLE table=default.Table_891be01aa2b540a7b16b407884b6de92 [id=f0491d89df70426c85ad61b15e43976d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.026624 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0576bc52f99440b9a6e7caf92309bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.026691 8113 tablet_service.cc:1505] Processing CreateTablet for tablet e0576bc52f99440b9a6e7caf92309bec (DEFAULT_TABLE table=default.Table_891be01aa2b540a7b16b407884b6de92 [id=f0491d89df70426c85ad61b15e43976d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.026961 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0576bc52f99440b9a6e7caf92309bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.027630 8544 tablet_bootstrap.cc:492] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.027709 8544 ts_tablet_manager.cc:1403] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20251028 02:59:59.028250 8544 raft_consensus.cc:359] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.028355 8544 raft_consensus.cc:385] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.028313 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 23790dc59186409b9cc9f4d6cd2d2e4e (DEFAULT_TABLE table=default.Table_b764dce6b08a49c5a196bd97d3297727 [id=b5d105fb03d04078a9e25db280a32c45]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.028386 8544 raft_consensus.cc:740] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.028513 8400 tablet_service.cc:1505] Processing CreateTablet for tablet f92acda752d5463aae30ce7e63f2f16f (DEFAULT_TABLE table=default.Table_acbe587ace4042f29f736a17a6d2530a [id=c5de3c9bf3524daf845f65158d3b14b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.028565 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23790dc59186409b9cc9f4d6cd2d2e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.028637 9788 raft_consensus.cc:493] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.028717 9788 raft_consensus.cc:515] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.028743 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92acda752d5463aae30ce7e63f2f16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.029088 9788 leader_election.cc:290] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.029513 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bca9ca8db1b491f9e6f7a564dc24ac1" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.029631 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bca9ca8db1b491f9e6f7a564dc24ac1" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.029702 7963 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 1de3acf65f744368ab31dc1c82ac554f (table default.Table_e5fa1e8f3825450685059a8a45433ef8 [id=739f0673774d472bbb62f176b70998bc]). Expected version 2 got 0
W20251028 02:59:59.030046 8083 leader_election.cc:343] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:59.030279 8083 leader_election.cc:343] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.030336 8083 leader_election.cc:304] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:59.030468 8256 tablet_service.cc:1505] Processing CreateTablet for tablet f92acda752d5463aae30ce7e63f2f16f (DEFAULT_TABLE table=default.Table_acbe587ace4042f29f736a17a6d2530a [id=c5de3c9bf3524daf845f65158d3b14b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.030639 9788 raft_consensus.cc:2749] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:59.030705 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92acda752d5463aae30ce7e63f2f16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.030823 8544 consensus_queue.cc:260] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.031145 8544 ts_tablet_manager.cc:1434] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:59.032264 8552 ts_tablet_manager.cc:1950] T aa0776d1a8ef43449309221b86ed3923 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.031203 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 1de3acf65f744368ab31dc1c82ac554f (table default.Table_e5fa1e8f3825450685059a8a45433ef8 [id=739f0673774d472bbb62f176b70998bc]). Expected version 2 got 0
I20251028 02:59:59.033109 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet aa0776d1a8ef43449309221b86ed3923 (table default.Table_6ad31a79d9f043628bdac1dd2d4c263c [id=4570d7ddf33c447fb40d6411ea47e52a]) successfully deleted
I20251028 02:59:59.033447 8552 tablet_replica.cc:333] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.033499 8551 ts_tablet_manager.cc:1916] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.033700 8552 raft_consensus.cc:2243] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.033895 8552 raft_consensus.cc:2272] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.034281 8552 ts_tablet_manager.cc:1916] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.037106 8552 ts_tablet_manager.cc:1929] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:59.037169 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet a76e16d10a174adda26b8ddd49ee528c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.037165 8544 tablet_bootstrap.cc:492] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.037405 8550 tablet_replica.cc:333] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.037539 8550 raft_consensus.cc:2243] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.037643 8550 raft_consensus.cc:2272] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.038326 8544 tablet_bootstrap.cc:654] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.038395 8545 tablet_bootstrap.cc:492] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.038480 8545 ts_tablet_manager.cc:1403] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 02:59:59.038996 8545 raft_consensus.cc:359] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.039104 8545 raft_consensus.cc:385] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.039135 8545 raft_consensus.cc:740] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.039253 8545 consensus_queue.cc:260] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.039543 8545 ts_tablet_manager.cc:1434] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.039649 8550 ts_tablet_manager.cc:1916] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.039759 8545 tablet_bootstrap.cc:492] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.039942 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 7a123640d3494675b58e878b6d4a77a6 (DEFAULT_TABLE table=default.Table_4d6d263beb42476ab582fad092f43a53 [id=49ad1222c8954dcdba7b6a7090bef51d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.040097 8551 ts_tablet_manager.cc:1929] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.040162 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a123640d3494675b58e878b6d4a77a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.040162 8551 log.cc:1199] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/aa0776d1a8ef43449309221b86ed3923
I20251028 02:59:59.040455 8551 ts_tablet_manager.cc:1950] T aa0776d1a8ef43449309221b86ed3923 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.040978 8545 tablet_bootstrap.cc:654] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.041023 8551 tablet_replica.cc:333] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.041163 8551 raft_consensus.cc:2243] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.041261 8551 raft_consensus.cc:2272] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.041615 8551 ts_tablet_manager.cc:1916] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.041890 8550 ts_tablet_manager.cc:1929] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.041949 8550 log.cc:1199] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a76e16d10a174adda26b8ddd49ee528c
I20251028 02:59:59.042168 8550 ts_tablet_manager.cc:1950] T a76e16d10a174adda26b8ddd49ee528c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:59.043102 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet aa0776d1a8ef43449309221b86ed3923 (table default.Table_6ad31a79d9f043628bdac1dd2d4c263c [id=4570d7ddf33c447fb40d6411ea47e52a]) successfully deleted
I20251028 02:59:59.045220 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]). Expected version 2 got 0
I20251028 02:59:59.045212 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 4594273181174e56beee7852128a394c (DEFAULT_TABLE table=default.Table_2f0fa5294def487f882830d451a7539e [id=ecc0e441fd6540d48c46bbef1f72b66d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.045274 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 2219dff427a84cab962dc7cd63086588 (DEFAULT_TABLE table=default.Table_35c25b81a8284119811feccb77fbf6a4 [id=1d651ee9479e4041b3be8470fcd2a31f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.045462 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4594273181174e56beee7852128a394c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.045852 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 7149926f0c4b4bd387ea8a1043367541 (DEFAULT_TABLE table=default.Table_bcfa6c66d227460abd8df8d84c443a3b [id=40be1cc147834cafa976568d43634f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.045926 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2219dff427a84cab962dc7cd63086588. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.046066 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7149926f0c4b4bd387ea8a1043367541. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.046674 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 05b6a671e514463bace6c3232700a110 (DEFAULT_TABLE table=default.Table_e29ed40d642b400192ea2dc90e9c28b0 [id=c7d8d868cfb14501aa49ba406a00f5ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.046916 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b6a671e514463bace6c3232700a110. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.047958 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a76e16d10a174adda26b8ddd49ee528c (table default.Table_273147e41d9f4c43b46e54c10ff21e09 [id=400620abf0584fadbb666f17e41477b0]) successfully deleted
I20251028 02:59:59.048272 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 2219dff427a84cab962dc7cd63086588 (DEFAULT_TABLE table=default.Table_35c25b81a8284119811feccb77fbf6a4 [id=1d651ee9479e4041b3be8470fcd2a31f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.048341 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 1f57930d5211413cb6cf4a693f7557d1 (DEFAULT_TABLE table=default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.048542 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2219dff427a84cab962dc7cd63086588. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.048341 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 05b6a671e514463bace6c3232700a110 (DEFAULT_TABLE table=default.Table_e29ed40d642b400192ea2dc90e9c28b0 [id=c7d8d868cfb14501aa49ba406a00f5ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.048844 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b6a671e514463bace6c3232700a110. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.050927 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 4594273181174e56beee7852128a394c (DEFAULT_TABLE table=default.Table_2f0fa5294def487f882830d451a7539e [id=ecc0e441fd6540d48c46bbef1f72b66d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.051152 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4594273181174e56beee7852128a394c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.051662 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f57930d5211413cb6cf4a693f7557d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.053017 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 7a123640d3494675b58e878b6d4a77a6 (DEFAULT_TABLE table=default.Table_4d6d263beb42476ab582fad092f43a53 [id=49ad1222c8954dcdba7b6a7090bef51d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.053258 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a123640d3494675b58e878b6d4a77a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.054136 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 7149926f0c4b4bd387ea8a1043367541 (DEFAULT_TABLE table=default.Table_bcfa6c66d227460abd8df8d84c443a3b [id=40be1cc147834cafa976568d43634f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.054426 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7149926f0c4b4bd387ea8a1043367541. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.054754 8544 tablet_bootstrap.cc:492] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.054859 8544 ts_tablet_manager.cc:1403] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.002s
I20251028 02:59:59.037824 8546 tablet_bootstrap.cc:492] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.055413 8544 raft_consensus.cc:359] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.055516 8544 raft_consensus.cc:385] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.055546 8544 raft_consensus.cc:740] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.055574 8546 ts_tablet_manager.cc:1403] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.000s
I20251028 02:59:59.038805 8259 tablet_service.cc:1505] Processing CreateTablet for tablet e0576bc52f99440b9a6e7caf92309bec (DEFAULT_TABLE table=default.Table_891be01aa2b540a7b16b407884b6de92 [id=f0491d89df70426c85ad61b15e43976d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.055648 8544 consensus_queue.cc:260] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.055852 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0576bc52f99440b9a6e7caf92309bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.055948 8544 ts_tablet_manager.cc:1434] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.056160 8544 tablet_bootstrap.cc:492] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.056123 8546 raft_consensus.cc:359] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.056228 8546 raft_consensus.cc:385] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.056257 8546 raft_consensus.cc:740] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.056371 8546 consensus_queue.cc:260] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.056620 8546 ts_tablet_manager.cc:1434] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.056804 8546 tablet_bootstrap.cc:492] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.057152 8544 tablet_bootstrap.cc:654] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.057804 8546 tablet_bootstrap.cc:654] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.057837 8551 ts_tablet_manager.cc:1929] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.057967 8551 log.cc:1199] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/49b46c13cb4e46388ed0a69d2ced9b42
I20251028 02:59:59.043591 9685 raft_consensus.cc:493] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.058207 9685 raft_consensus.cc:515] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.058260 8551 ts_tablet_manager.cc:1950] T 49b46c13cb4e46388ed0a69d2ced9b42 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.058557 9685 leader_election.cc:290] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:59.044579 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 7a123640d3494675b58e878b6d4a77a6 (DEFAULT_TABLE table=default.Table_4d6d263beb42476ab582fad092f43a53 [id=49ad1222c8954dcdba7b6a7090bef51d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.044987 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 7149926f0c4b4bd387ea8a1043367541 (DEFAULT_TABLE table=default.Table_bcfa6c66d227460abd8df8d84c443a3b [id=40be1cc147834cafa976568d43634f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.045192 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 2219dff427a84cab962dc7cd63086588 (DEFAULT_TABLE table=default.Table_35c25b81a8284119811feccb77fbf6a4 [id=1d651ee9479e4041b3be8470fcd2a31f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.058951 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a123640d3494675b58e878b6d4a77a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.058940 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 49b46c13cb4e46388ed0a69d2ced9b42 (table default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]) successfully deleted
W20251028 02:59:59.059396 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet e369dbabc2c04ad5ab17bbd4225127c1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e369dbabc2c04ad5ab17bbd4225127c1 already in progress: creating tablet
I20251028 02:59:59.059621 8551 tablet_replica.cc:333] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.059737 8551 raft_consensus.cc:2243] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.059854 8551 raft_consensus.cc:2272] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.060257 8551 ts_tablet_manager.cc:1916] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.060797 8552 log.cc:1199] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/56c8969da9dc464d9b463205e3a27965
I20251028 02:59:59.046257 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 4594273181174e56beee7852128a394c (DEFAULT_TABLE table=default.Table_2f0fa5294def487f882830d451a7539e [id=ecc0e441fd6540d48c46bbef1f72b66d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.048946 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 05b6a671e514463bace6c3232700a110 (DEFAULT_TABLE table=default.Table_e29ed40d642b400192ea2dc90e9c28b0 [id=c7d8d868cfb14501aa49ba406a00f5ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.049170 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 1f57930d5211413cb6cf4a693f7557d1 (DEFAULT_TABLE table=default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.052542 8553 consensus_queue.cc:1048] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.061069 8552 ts_tablet_manager.cc:1950] T 56c8969da9dc464d9b463205e3a27965 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.061721 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1de3acf65f744368ab31dc1c82ac554f" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:59.061858 8145 raft_consensus.cc:2468] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:59.062009 8545 tablet_bootstrap.cc:492] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.062093 8545 ts_tablet_manager.cc:1403] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20251028 02:59:59.062389 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 13fb883843d141f09aa90e66f9b7bf1f (table default.Table_2547f4c4cdbd4d7f9ec732da77dc545a [id=c9c59047d50a4419a9f16431bce05cd1]). Expected version 1 got 0
I20251028 02:59:59.062587 8545 raft_consensus.cc:359] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.062690 8545 raft_consensus.cc:385] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.062721 8545 raft_consensus.cc:740] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.062852 8545 consensus_queue.cc:260] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.062918 8218 leader_election.cc:304] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:59.063091 8553 raft_consensus.cc:2804] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:59.063143 8553 raft_consensus.cc:493] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:59.063184 8553 raft_consensus.cc:3060] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.063166 8545 ts_tablet_manager.cc:1434] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.063445 8545 tablet_bootstrap.cc:492] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.063892 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2219dff427a84cab962dc7cd63086588. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.064210 8553 raft_consensus.cc:515] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.064529 8545 tablet_bootstrap.cc:654] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.064590 8553 leader_election.cc:290] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:59.066381 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7149926f0c4b4bd387ea8a1043367541. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.067160 8544 tablet_bootstrap.cc:492] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.067246 8544 ts_tablet_manager.cc:1403] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 02:59:59.067831 8544 raft_consensus.cc:359] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.067934 8544 raft_consensus.cc:385] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.067962 8544 raft_consensus.cc:740] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.068117 7963 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 13fb883843d141f09aa90e66f9b7bf1f (table default.Table_2547f4c4cdbd4d7f9ec732da77dc545a [id=c9c59047d50a4419a9f16431bce05cd1]). Expected version 1 got 0
I20251028 02:59:59.068185 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1de3acf65f744368ab31dc1c82ac554f" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.068290 8416 raft_consensus.cc:2468] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:59.068392 8544 consensus_queue.cc:260] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.068671 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4594273181174e56beee7852128a394c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.068722 8544 ts_tablet_manager.cc:1434] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.068899 8544 tablet_bootstrap.cc:492] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.069499 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 900973d832c24b62a4554ae9a219cfd8 (table default.Table_f0dafbe2e0a84541a79cf6c887d59619 [id=c4aadbebe43f451ea22467feaa6dafbb]). Expected version 1 got 0
I20251028 02:59:59.069612 9788 raft_consensus.cc:493] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.069693 9788 raft_consensus.cc:515] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.070109 9788 leader_election.cc:290] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:59.070858 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b6a671e514463bace6c3232700a110. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.071684 8546 tablet_bootstrap.cc:492] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.071789 8546 ts_tablet_manager.cc:1403] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.001s
I20251028 02:59:59.072279 8546 raft_consensus.cc:359] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.072377 8546 raft_consensus.cc:385] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.072417 8546 raft_consensus.cc:740] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.072543 8546 consensus_queue.cc:260] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.072643 8544 tablet_bootstrap.cc:654] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.072861 8546 ts_tablet_manager.cc:1434] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:59.072976 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f57930d5211413cb6cf4a693f7557d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.073060 8546 tablet_bootstrap.cc:492] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.074039 8546 tablet_bootstrap.cc:654] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.075181 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d5fcc9b539b46458cd8cbfc30cbcd0a" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.075181 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]). Expected version 2 got 0
W20251028 02:59:59.075524 8083 leader_election.cc:343] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.075673 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d5fcc9b539b46458cd8cbfc30cbcd0a" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.076396 9595 raft_consensus.cc:493] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.076480 9595 raft_consensus.cc:515] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.076536 9639 raft_consensus.cc:493] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.076617 9639 raft_consensus.cc:515] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.076831 9595 leader_election.cc:290] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.076937 9639 leader_election.cc:290] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 02:59:59.077354 8083 leader_election.cc:343] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.077427 8083 leader_election.cc:304] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:59.077564 9401 raft_consensus.cc:2749] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:59.077567 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1de3acf65f744368ab31dc1c82ac554f" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:59.077669 8416 raft_consensus.cc:3060] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.077788 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1de3acf65f744368ab31dc1c82ac554f" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:59.077893 8145 raft_consensus.cc:3060] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.078037 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b46554b0df894beb8141126aa7ee7b8e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:59.078238 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "900973d832c24b62a4554ae9a219cfd8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:59.078332 8143 raft_consensus.cc:2468] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:59.078709 8218 leader_election.cc:304] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:59.078737 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a with delete_type TABLET_DATA_DELETED (replaced by tablet 4eaa472029a94052abb49241b7622d0f) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.078781 8416 raft_consensus.cc:2468] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:59.078909 8550 tablet_replica.cc:333] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.079012 8145 raft_consensus.cc:2468] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:59.079068 8550 raft_consensus.cc:2243] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.079128 8553 raft_consensus.cc:2804] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:59.079165 8550 raft_consensus.cc:2272] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.079200 8553 raft_consensus.cc:493] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:59.079239 8553 raft_consensus.cc:3060] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.079641 8550 ts_tablet_manager.cc:1916] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.080276 8553 raft_consensus.cc:515] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.080418 8220 leader_election.cc:304] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:59.080583 9595 raft_consensus.cc:2804] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:59.080653 8553 leader_election.cc:290] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.080658 9595 raft_consensus.cc:697] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:59.081204 9595 consensus_queue.cc:237] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.081665 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b46554b0df894beb8141126aa7ee7b8e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.082229 8551 ts_tablet_manager.cc:1929] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:59.082298 8551 log.cc:1199] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/56c8969da9dc464d9b463205e3a27965
I20251028 02:59:59.082497 8550 ts_tablet_manager.cc:1929] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.082546 8551 ts_tablet_manager.cc:1950] T 56c8969da9dc464d9b463205e3a27965 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.082571 8550 log.cc:1199] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/1d5fcc9b539b46458cd8cbfc30cbcd0a
I20251028 02:59:59.082782 8550 ts_tablet_manager.cc:1950] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:59.083086 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 900973d832c24b62a4554ae9a219cfd8 (table default.Table_f0dafbe2e0a84541a79cf6c887d59619 [id=c4aadbebe43f451ea22467feaa6dafbb]). Expected version 1 got 0
I20251028 02:59:59.083209 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:59.083412 8356 leader_election.cc:304] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:59.083458 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a (table default.Table_059639c7191541caa3dbb30b333d579f [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
I20251028 02:59:59.083710 9639 raft_consensus.cc:2749] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:59.083758 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "900973d832c24b62a4554ae9a219cfd8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.083856 8416 raft_consensus.cc:2468] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:59.083248 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 56c8969da9dc464d9b463205e3a27965 (table default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]) successfully deleted
I20251028 02:59:59.084450 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 56c8969da9dc464d9b463205e3a27965 (table default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]) successfully deleted
I20251028 02:59:59.084494 8552 tablet_replica.cc:333] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.084641 8552 raft_consensus.cc:2243] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.084746 8552 raft_consensus.cc:2272] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.084945 8551 tablet_replica.cc:333] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.085079 8551 raft_consensus.cc:2243] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:59.085136 8552 ts_tablet_manager.cc:1916] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.085251 8551 raft_consensus.cc:2272] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.085618 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "900973d832c24b62a4554ae9a219cfd8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:59.085716 8416 raft_consensus.cc:3060] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.086300 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "900973d832c24b62a4554ae9a219cfd8" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:59.086406 8145 raft_consensus.cc:3060] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.086633 8416 raft_consensus.cc:2468] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:59.087472 8145 raft_consensus.cc:2468] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:59.087827 8220 leader_election.cc:304] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:59.087823 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]). Expected version 2 got 0
I20251028 02:59:59.088043 9595 raft_consensus.cc:2804] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:59.087950 7960 catalog_manager.cc:5649] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:59.088117 9595 raft_consensus.cc:697] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:59.088171 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 1de3acf65f744368ab31dc1c82ac554f (table default.Table_e5fa1e8f3825450685059a8a45433ef8 [id=739f0673774d472bbb62f176b70998bc]). Expected version 2 got 0
I20251028 02:59:59.088229 9595 consensus_queue.cc:237] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.093966 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 1f57930d5211413cb6cf4a693f7557d1 (DEFAULT_TABLE table=default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.094275 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f57930d5211413cb6cf4a693f7557d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.094712 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 88cb16241e364f56b251d88079936c7b (DEFAULT_TABLE table=default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.094949 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88cb16241e364f56b251d88079936c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.096364 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet aaaec176a3884a79a55bb74397a7dc9c with delete_type TABLET_DATA_DELETED (replaced by tablet 2bf100407b034626b4a9f7e2340f9cf0) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.096983 8552 ts_tablet_manager.cc:1929] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.097143 8552 log.cc:1199] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/0868c69e28a840b4ab9dad61d50983ec
I20251028 02:59:59.097666 8552 ts_tablet_manager.cc:1950] T 0868c69e28a840b4ab9dad61d50983ec P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.097609 7960 catalog_manager.cc:5649] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:59.098146 8545 tablet_bootstrap.cc:492] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.098253 8545 ts_tablet_manager.cc:1403] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.000s
I20251028 02:59:59.098732 8545 raft_consensus.cc:359] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.098845 8545 raft_consensus.cc:385] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.098882 8545 raft_consensus.cc:740] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.099016 8545 consensus_queue.cc:260] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.099282 8545 ts_tablet_manager.cc:1434] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.099679 9685 consensus_queue.cc:1048] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.099839 8545 tablet_bootstrap.cc:492] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.100311 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 900973d832c24b62a4554ae9a219cfd8 (table default.Table_f0dafbe2e0a84541a79cf6c887d59619 [id=c4aadbebe43f451ea22467feaa6dafbb]). Expected version 1 got 0
I20251028 02:59:59.098855 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 0868c69e28a840b4ab9dad61d50983ec (table default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]) successfully deleted
I20251028 02:59:59.101549 8552 tablet_replica.cc:333] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.101770 8552 raft_consensus.cc:2243] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.101927 8552 raft_consensus.cc:2272] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.102633 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 83059074c365440b84e39cf7a5553eb2 with delete_type TABLET_DATA_DELETED (replaced by tablet c120a3d56f6b41c28c7f7dcd84359a60) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.102710 8545 tablet_bootstrap.cc:654] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.098141 8551 ts_tablet_manager.cc:1916] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.104010 8552 ts_tablet_manager.cc:1916] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.107621 8552 ts_tablet_manager.cc:1929] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.107790 8552 log.cc:1199] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/83059074c365440b84e39cf7a5553eb2
I20251028 02:59:59.108673 7963 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:59.109573 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 88cb16241e364f56b251d88079936c7b (DEFAULT_TABLE table=default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.109907 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88cb16241e364f56b251d88079936c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.110230 8113 tablet_service.cc:1505] Processing CreateTablet for tablet e97c2e9bb8284a5997c62b43dc52f32b (DEFAULT_TABLE table=default.Table_7072983821f0413c95a0ff43b50ada34 [id=70a7f2b28a014abaaa7615f54bbdae2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.110528 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97c2e9bb8284a5997c62b43dc52f32b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.113004 8400 tablet_service.cc:1505] Processing CreateTablet for tablet e97c2e9bb8284a5997c62b43dc52f32b (DEFAULT_TABLE table=default.Table_7072983821f0413c95a0ff43b50ada34 [id=70a7f2b28a014abaaa7615f54bbdae2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.113364 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97c2e9bb8284a5997c62b43dc52f32b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.115329 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 4a57e122bbb94a9fac8c8254ab73a21f (DEFAULT_TABLE table=default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.115604 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a57e122bbb94a9fac8c8254ab73a21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.115705 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 4a57e122bbb94a9fac8c8254ab73a21f (DEFAULT_TABLE table=default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.115931 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a57e122bbb94a9fac8c8254ab73a21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.118021 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 2b6f62b942a94af783c026a043cc60a3 (DEFAULT_TABLE table=default.Table_6c4478bf2de14c3c824d9ab70557b237 [id=82ae3d70b0b544d89bd8e0ff1717f396]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.118257 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b6f62b942a94af783c026a043cc60a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.118503 8544 tablet_bootstrap.cc:492] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.118515 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 2b6f62b942a94af783c026a043cc60a3 (DEFAULT_TABLE table=default.Table_6c4478bf2de14c3c824d9ab70557b237 [id=82ae3d70b0b544d89bd8e0ff1717f396]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.118606 8544 ts_tablet_manager.cc:1403] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.000s
I20251028 02:59:59.118753 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b6f62b942a94af783c026a043cc60a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.119132 8544 raft_consensus.cc:359] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.119233 8544 raft_consensus.cc:385] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.119261 8544 raft_consensus.cc:740] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.119413 8544 consensus_queue.cc:260] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.119737 8544 ts_tablet_manager.cc:1434] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 02:59:59.119907 8544 tablet_bootstrap.cc:492] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.120940 8544 tablet_bootstrap.cc:654] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.121004 9840 raft_consensus.cc:493] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.121086 9840 raft_consensus.cc:515] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.122308 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "240b7da6f46c4c5ab8087ab1dd6275c5" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:59.122418 8143 raft_consensus.cc:2468] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:59.122135 9840 leader_election.cc:290] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.108611 8552 ts_tablet_manager.cc:1950] T 83059074c365440b84e39cf7a5553eb2 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.109377 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 88cb16241e364f56b251d88079936c7b (DEFAULT_TABLE table=default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.122830 8354 leader_election.cc:304] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:59.122871 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88cb16241e364f56b251d88079936c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.123092 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:59.123361 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 83059074c365440b84e39cf7a5553eb2 (table default.Table_13dbc4fadcfd41c8945172798836fb62 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]) successfully deleted
I20251028 02:59:59.123539 9840 raft_consensus.cc:2804] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:59.123603 9840 raft_consensus.cc:493] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:59.123656 9840 raft_consensus.cc:3060] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.124603 9840 raft_consensus.cc:515] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.125257 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "240b7da6f46c4c5ab8087ab1dd6275c5" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:59.125373 8143 raft_consensus.cc:3060] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.114193 8249 tablet_service.cc:1505] Processing CreateTablet for tablet e97c2e9bb8284a5997c62b43dc52f32b (DEFAULT_TABLE table=default.Table_7072983821f0413c95a0ff43b50ada34 [id=70a7f2b28a014abaaa7615f54bbdae2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.125671 9840 leader_election.cc:290] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.125878 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97c2e9bb8284a5997c62b43dc52f32b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.126336 8552 tablet_replica.cc:333] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.126392 8143 raft_consensus.cc:2468] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:59.126453 8551 ts_tablet_manager.cc:1929] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.126513 8551 log.cc:1199] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/0868c69e28a840b4ab9dad61d50983ec
I20251028 02:59:59.126686 8354 leader_election.cc:304] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:59.126885 9840 raft_consensus.cc:2804] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:59.126958 9840 raft_consensus.cc:697] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:59.127086 9840 consensus_queue.cc:237] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.127789 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:59.115000 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 4a57e122bbb94a9fac8c8254ab73a21f (DEFAULT_TABLE table=default.Table_46d5cdd6882c46da97c21a768a91d858 [id=091f678d62af4a0792dfccedb3db996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.128497 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a57e122bbb94a9fac8c8254ab73a21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.129030 8552 raft_consensus.cc:2243] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.129150 8552 raft_consensus.cc:2272] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.129225 8545 tablet_bootstrap.cc:492] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.129310 8545 ts_tablet_manager.cc:1403] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.002s
I20251028 02:59:59.129666 8552 ts_tablet_manager.cc:1916] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.129977 8545 raft_consensus.cc:359] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.130077 8545 raft_consensus.cc:385] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.130106 8545 raft_consensus.cc:740] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.130244 8545 consensus_queue.cc:260] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.116050 8546 tablet_bootstrap.cc:492] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.130475 8546 ts_tablet_manager.cc:1403] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.057s user 0.000s sys 0.002s
I20251028 02:59:59.130851 8545 ts_tablet_manager.cc:1434] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:59.130987 8546 raft_consensus.cc:359] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.131083 8546 raft_consensus.cc:385] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.131112 8546 raft_consensus.cc:740] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.131217 8546 consensus_queue.cc:260] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.131336 8545 tablet_bootstrap.cc:492] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.131528 8546 ts_tablet_manager.cc:1434] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.131707 8546 tablet_bootstrap.cc:492] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.132409 8545 tablet_bootstrap.cc:654] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.132838 8546 tablet_bootstrap.cc:654] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.133206 8551 ts_tablet_manager.cc:1950] T 0868c69e28a840b4ab9dad61d50983ec P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.117034 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 2b6f62b942a94af783c026a043cc60a3 (DEFAULT_TABLE table=default.Table_6c4478bf2de14c3c824d9ab70557b237 [id=82ae3d70b0b544d89bd8e0ff1717f396]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.133841 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b6f62b942a94af783c026a043cc60a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.133846 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 0868c69e28a840b4ab9dad61d50983ec (table default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]) successfully deleted
I20251028 02:59:59.134102 8551 tablet_replica.cc:333] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.134233 8551 raft_consensus.cc:2243] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.134328 8551 raft_consensus.cc:2272] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.134792 8551 ts_tablet_manager.cc:1916] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.137167 8551 ts_tablet_manager.cc:1929] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.137243 8551 log.cc:1199] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3b43da9580244346ae512a7bc9b506d3
I20251028 02:59:59.137197 7961 catalog_manager.cc:5649] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:59.137449 8551 ts_tablet_manager.cc:1950] T 3b43da9580244346ae512a7bc9b506d3 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.137557 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]). Expected version 2 got 0
I20251028 02:59:59.137773 9685 raft_consensus.cc:493] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.137856 9685 raft_consensus.cc:515] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.138065 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3b43da9580244346ae512a7bc9b506d3 (table default.Table_a00976e88f0147c5b01560dba597ea82 [id=7244d96a49d34411bb8f9f7416904ae3]) successfully deleted
I20251028 02:59:59.138217 9685 leader_election.cc:290] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.122264 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "240b7da6f46c4c5ab8087ab1dd6275c5" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.138945 8273 raft_consensus.cc:2468] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:59.139719 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_35aa9cb3afdc4a149341fa51b840258f" } modify_external_catalogs: true
I20251028 02:59:59.139839 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_35aa9cb3afdc4a149341fa51b840258f" } modify_external_catalogs: true
I20251028 02:59:59.140239 8552 ts_tablet_manager.cc:1929] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.140305 8552 log.cc:1199] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/aaaec176a3884a79a55bb74397a7dc9c
I20251028 02:59:59.140528 8552 ts_tablet_manager.cc:1950] T aaaec176a3884a79a55bb74397a7dc9c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.140686 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c612bc752847c2be0547cedfd665ae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:59.140718 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7e5a0a5655db40d8957f85af13a091b5" } modify_external_catalogs: true
I20251028 02:59:59.140815 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7e5a0a5655db40d8957f85af13a091b5" } modify_external_catalogs: true
I20251028 02:59:59.140820 8143 raft_consensus.cc:2468] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:59.140851 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1f11d33cf9ac4f11bc540fa516f55d2b" } modify_external_catalogs: true
I20251028 02:59:59.140923 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1f11d33cf9ac4f11bc540fa516f55d2b" } modify_external_catalogs: true
I20251028 02:59:59.141573 8218 leader_election.cc:304] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:59.141762 9685 raft_consensus.cc:2804] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:59.141816 9685 raft_consensus.cc:493] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:59.141853 9685 raft_consensus.cc:3060] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.142197 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c612bc752847c2be0547cedfd665ae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.142318 8419 raft_consensus.cc:2468] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:59.142827 9685 raft_consensus.cc:515] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.142944 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "240b7da6f46c4c5ab8087ab1dd6275c5" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:59.143051 8271 raft_consensus.cc:3060] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.143225 9685 leader_election.cc:290] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.143973 8271 raft_consensus.cc:2468] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:59.145478 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:59.145944 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c612bc752847c2be0547cedfd665ae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:59.146088 8143 raft_consensus.cc:3060] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.146700 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet aaaec176a3884a79a55bb74397a7dc9c (table default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262]) successfully deleted
I20251028 02:59:59.146718 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2c04cfc8df68467d9ed65a7499d4e827"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:59.147042 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b4c612bc752847c2be0547cedfd665ae" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
W20251028 02:59:59.147130 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2c04cfc8df68467d9ed65a7499d4e827 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.147156 8419 raft_consensus.cc:3060] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.147149 8143 raft_consensus.cc:2468] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:59.148468 8218 leader_election.cc:304] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 02:59:59.148682 9685 raft_consensus.cc:2804] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:59.148761 9685 raft_consensus.cc:697] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:59.148882 9685 consensus_queue.cc:237] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.149000 8546 tablet_bootstrap.cc:492] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.149075 8546 ts_tablet_manager.cc:1403] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 02:59:59.149549 8546 raft_consensus.cc:359] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.149653 8546 raft_consensus.cc:385] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.149693 8546 raft_consensus.cc:740] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.149838 8546 consensus_queue.cc:260] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.149906 8419 raft_consensus.cc:2468] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:59.150112 8546 ts_tablet_manager.cc:1434] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.150274 8546 tablet_bootstrap.cc:492] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.153450 7968 catalog_manager.cc:5649] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:59.155197 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a67c21155a494d2cab8af8ca6668c1d8
I20251028 02:59:59.155365 8544 tablet_bootstrap.cc:492] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.155467 8544 ts_tablet_manager.cc:1403] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20251028 02:59:59.155786 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 6dd563e13e624d4bb88950b46d480876 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:59.156028 8544 raft_consensus.cc:359] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.156144 8544 raft_consensus.cc:385] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.156178 8544 raft_consensus.cc:740] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.156299 8544 consensus_queue.cc:260] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.156405 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 302e0ed455854eb2b661afc2f2b6921d
I20251028 02:59:59.156378 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet a67c21155a494d2cab8af8ca6668c1d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.156611 8544 ts_tablet_manager.cc:1434] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.156788 8544 tablet_bootstrap.cc:492] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.156975 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4b787f8f024745b297b2ebaa456cbcbf
I20251028 02:59:59.157955 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b787f8f024745b297b2ebaa456cbcbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.158121 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet a67c21155a494d2cab8af8ca6668c1d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:59.159075 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4b787f8f024745b297b2ebaa456cbcbf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b787f8f024745b297b2ebaa456cbcbf already in progress: creating tablet
I20251028 02:59:59.159667 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b787f8f024745b297b2ebaa456cbcbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.159832 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b787f8f024745b297b2ebaa456cbcbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.159958 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 302e0ed455854eb2b661afc2f2b6921d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.160034 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 302e0ed455854eb2b661afc2f2b6921d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.160091 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet a67c21155a494d2cab8af8ca6668c1d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.160306 8546 tablet_bootstrap.cc:654] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:59.160533 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a67c21155a494d2cab8af8ca6668c1d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a67c21155a494d2cab8af8ca6668c1d8 already in progress: creating tablet
W20251028 02:59:59.160570 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a67c21155a494d2cab8af8ca6668c1d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a67c21155a494d2cab8af8ca6668c1d8 already in progress: creating tablet
I20251028 02:59:59.161166 8544 tablet_bootstrap.cc:654] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.161680 8545 tablet_bootstrap.cc:492] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.161787 8545 ts_tablet_manager.cc:1403] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
W20251028 02:59:59.162007 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4b787f8f024745b297b2ebaa456cbcbf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b787f8f024745b297b2ebaa456cbcbf already in progress: creating tablet
I20251028 02:59:59.162322 8545 raft_consensus.cc:359] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.162423 8545 raft_consensus.cc:385] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.162451 8545 raft_consensus.cc:740] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.162561 8545 consensus_queue.cc:260] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
W20251028 02:59:59.162694 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 302e0ed455854eb2b661afc2f2b6921d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 302e0ed455854eb2b661afc2f2b6921d already in progress: creating tablet
I20251028 02:59:59.162845 8545 ts_tablet_manager.cc:1434] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.163040 8545 tablet_bootstrap.cc:492] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.164120 8545 tablet_bootstrap.cc:654] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.164855 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 302e0ed455854eb2b661afc2f2b6921d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.164902 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e4eec90a2fa412bb0954805f5b8d832 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.165424 8550 tablet_replica.cc:333] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.165386 8551 tablet_replica.cc:333] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.165580 8550 raft_consensus.cc:2243] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.165680 8550 raft_consensus.cc:2272] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.165678 8551 raft_consensus.cc:2243] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.165778 8551 raft_consensus.cc:2272] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:59.166096 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 302e0ed455854eb2b661afc2f2b6921d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 302e0ed455854eb2b661afc2f2b6921d already in progress: creating tablet
I20251028 02:59:59.166268 8551 ts_tablet_manager.cc:1916] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.168460 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_cff7897b7440432c96a505a493a0b149" } modify_external_catalogs: true
I20251028 02:59:59.168558 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5d25b16481c84800a7dbb53c61132cbb" } new_table_name: "default.Table_2e728b4bb7394afaa5a9fd93d7f2739e" modify_external_catalogs: true
I20251028 02:59:59.168583 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_cff7897b7440432c96a505a493a0b149" } modify_external_catalogs: true
I20251028 02:59:59.168746 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f42e5ece3b0049de9c9f81ea3d595d19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:59.168856 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_08f544277a764846bfb2cbda994300d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.169143 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f42e5ece3b0049de9c9f81ea3d595d19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.169255 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_bb04323fd765408aa725406848f20bef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.169543 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb04323fd765408aa725406848f20bef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.169770 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 4902bc3398754802a5c571036cd19fa7 with delete_type TABLET_DATA_DELETED (replaced by tablet b42f039c30a048e299c39bcf44033125) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:59.169157 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08f544277a764846bfb2cbda994300d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.170867 8552 tablet_replica.cc:333] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.171891 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_21c8b9b2691f402f931349d798c2b2c4" } new_table_name: "default.Table_8acb4514466d406a89de3678169c6180" modify_external_catalogs: true
I20251028 02:59:59.171973 9839 consensus_queue.cc:1048] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.172474 8552 raft_consensus.cc:2243] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.172600 8552 raft_consensus.cc:2272] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.173179 8552 ts_tablet_manager.cc:1916] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.177879 8551 ts_tablet_manager.cc:1929] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.178062 8551 log.cc:1199] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/83059074c365440b84e39cf7a5553eb2
I20251028 02:59:59.178643 8551 ts_tablet_manager.cc:1950] T 83059074c365440b84e39cf7a5553eb2 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.179698 8552 ts_tablet_manager.cc:1929] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.180687 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_70cf1ece6d88419b952556cc05782215"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.181114 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_70cf1ece6d88419b952556cc05782215 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.181329 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_639df418295f428b86043a57b3c6ea58" } modify_external_catalogs: true
I20251028 02:59:59.181686 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a6aa4919568f4f3fbfd16ce34c062d3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.182019 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a6aa4919568f4f3fbfd16ce34c062d3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.182144 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 83059074c365440b84e39cf7a5553eb2 (table default.Table_13dbc4fadcfd41c8945172798836fb62 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]) successfully deleted
W20251028 02:59:59.183125 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a67c21155a494d2cab8af8ca6668c1d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a67c21155a494d2cab8af8ca6668c1d8 already in progress: creating tablet
I20251028 02:59:59.183615 8552 log.cc:1199] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4902bc3398754802a5c571036cd19fa7
W20251028 02:59:59.184410 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4b787f8f024745b297b2ebaa456cbcbf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b787f8f024745b297b2ebaa456cbcbf already in progress: creating tablet
I20251028 02:59:59.185412 8550 ts_tablet_manager.cc:1916] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:59.186762 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5c1994484abf4b6fbbc3f47527558cdb consensus state has no leader
I20251028 02:59:59.187043 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3da0f96b500a4ff7b76f7aaa052a52a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.187450 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3da0f96b500a4ff7b76f7aaa052a52a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.187913 8550 ts_tablet_manager.cc:1929] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.187981 8550 log.cc:1199] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9e4eec90a2fa412bb0954805f5b8d832
I20251028 02:59:59.188191 8550 ts_tablet_manager.cc:1950] T 9e4eec90a2fa412bb0954805f5b8d832 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:59.188292 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_639df418295f428b86043a57b3c6ea58" } modify_external_catalogs: true
W20251028 02:59:59.189580 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 302e0ed455854eb2b661afc2f2b6921d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 302e0ed455854eb2b661afc2f2b6921d already in progress: creating tablet
I20251028 02:59:59.189839 9839 consensus_queue.cc:1048] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.190174 8552 ts_tablet_manager.cc:1950] T 4902bc3398754802a5c571036cd19fa7 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.190981 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c7e0f5beff234f17b69504efc678924d" } new_table_name: "default.Table_e204f02c01f848fda844db72d743be8e" modify_external_catalogs: true
I20251028 02:59:59.191188 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4902bc3398754802a5c571036cd19fa7 (table default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]) successfully deleted
I20251028 02:59:59.192239 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a4a285cc359344a59f28244ff43d0da4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.192605 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4a285cc359344a59f28244ff43d0da4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.193732 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b12ebf9b73404d62b93a0eefa8d10b53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.194068 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b12ebf9b73404d62b93a0eefa8d10b53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 02:59:59.197254 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4de961f542d84fad8cbce64ce99df114 consensus state has no leader
I20251028 02:59:59.197497 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_121342cd2f774cce941d56a09004daae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.197861 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_121342cd2f774cce941d56a09004daae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.198963 8419 raft_consensus.cc:3060] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.200604 8546 tablet_bootstrap.cc:492] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.200690 8546 ts_tablet_manager.cc:1403] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.050s user 0.000s sys 0.002s
I20251028 02:59:59.201066 9733 tablet.cc:1722] T b46554b0df894beb8141126aa7ee7b8e P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.201162 8546 raft_consensus.cc:359] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.201265 8546 raft_consensus.cc:385] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.201292 8546 raft_consensus.cc:740] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.201399 8546 consensus_queue.cc:260] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.201659 8546 ts_tablet_manager.cc:1434] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:59.201836 8546 tablet_bootstrap.cc:492] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.201845 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9e4eec90a2fa412bb0954805f5b8d832 (table default.Table_90d5c8bb11484e5ab7648f8cb5d5c4e0 [id=a4f9578fb0c146a7af9769f101d57da5]) successfully deleted
I20251028 02:59:59.202859 8546 tablet_bootstrap.cc:654] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.212956 8544 tablet_bootstrap.cc:492] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.213248 8544 ts_tablet_manager.cc:1403] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.057s user 0.002s sys 0.000s
I20251028 02:59:59.213529 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_62c4ca6858344881a02383c7919d85cb" } new_table_name: "default.Table_8e38a9fe73744973bb30019aa837de23" modify_external_catalogs: true
I20251028 02:59:59.213963 8544 raft_consensus.cc:359] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.214053 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a35599c912a74f959526ab6945536e2d" } new_table_name: "default.Table_73967d33588f4f02ab5e0010826ec5f6" modify_external_catalogs: true
I20251028 02:59:59.214201 8544 raft_consensus.cc:385] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.214296 8544 raft_consensus.cc:740] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.214495 8544 consensus_queue.cc:260] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.214869 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 763022e76e23448a8ae0c9320eb8fffc
I20251028 02:59:59.215291 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet f39ed19223fa4268a01aaf4c7762d86e from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:59.215497 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 763022e76e23448a8ae0c9320eb8fffc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.215814 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 763022e76e23448a8ae0c9320eb8fffc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.216466 8544 ts_tablet_manager.cc:1434] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:59.216602 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8dab70ca2d9f4b5fba338cb337b0b396
I20251028 02:59:59.217051 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dab70ca2d9f4b5fba338cb337b0b396 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:59.217097 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1d3e422c7ae64b25960714ecc926046a consensus state has no leader
I20251028 02:59:59.217092 8544 tablet_bootstrap.cc:492] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.217792 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 763022e76e23448a8ae0c9320eb8fffc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.217944 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dab70ca2d9f4b5fba338cb337b0b396 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:59.220846 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 763022e76e23448a8ae0c9320eb8fffc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 763022e76e23448a8ae0c9320eb8fffc already in progress: creating tablet
W20251028 02:59:59.220726 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 763022e76e23448a8ae0c9320eb8fffc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 763022e76e23448a8ae0c9320eb8fffc already in progress: creating tablet
I20251028 02:59:59.221100 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dab70ca2d9f4b5fba338cb337b0b396 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:59.221495 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 8dab70ca2d9f4b5fba338cb337b0b396 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8dab70ca2d9f4b5fba338cb337b0b396 already in progress: creating tablet
W20251028 02:59:59.221544 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 8dab70ca2d9f4b5fba338cb337b0b396 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8dab70ca2d9f4b5fba338cb337b0b396 already in progress: creating tablet
W20251028 02:59:59.222101 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 763022e76e23448a8ae0c9320eb8fffc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 763022e76e23448a8ae0c9320eb8fffc already in progress: creating tablet
W20251028 02:59:59.222265 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 8dab70ca2d9f4b5fba338cb337b0b396 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8dab70ca2d9f4b5fba338cb337b0b396 already in progress: creating tablet
I20251028 02:59:59.225401 8553 consensus_queue.cc:1048] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:59.226696 9849 log.cc:927] Time spent T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20251028 02:59:59.228055 8545 tablet_bootstrap.cc:492] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.228156 8545 ts_tablet_manager.cc:1403] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.000s
I20251028 02:59:59.228634 8545 raft_consensus.cc:359] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.228744 8545 raft_consensus.cc:385] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.228780 8545 raft_consensus.cc:740] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.228906 8545 consensus_queue.cc:260] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.229205 8545 ts_tablet_manager.cc:1434] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.229403 8545 tablet_bootstrap.cc:492] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.229595 8544 tablet_bootstrap.cc:654] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.230403 8545 tablet_bootstrap.cc:654] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.232043 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f39ed19223fa4268a01aaf4c7762d86e
I20251028 02:59:59.232367 7962 catalog_manager.cc:6719] ReplaceTablet: tablet f39ed19223fa4268a01aaf4c7762d86e deleted and replaced by tablet b6a2791098934a9c8aa21d8db88f2c9e
W20251028 02:59:59.232388 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f6818c3f2dfa48448bb2d9b38ed0dc19 consensus state has no leader
I20251028 02:59:59.232640 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 5059e1ed0b494788bfd571987160b0b4 from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:59.233714 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d195958488bf401c8668116a97bf0f58 consensus state has no leader
I20251028 02:59:59.233819 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet f39ed19223fa4268a01aaf4c7762d86e with delete_type TABLET_DATA_DELETED (replaced by tablet b6a2791098934a9c8aa21d8db88f2c9e) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.234084 8545 tablet_bootstrap.cc:492] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.234283 8545 ts_tablet_manager.cc:1403] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 02:59:59.234402 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet f39ed19223fa4268a01aaf4c7762d86e with delete_type TABLET_DATA_DELETED (replaced by tablet b6a2791098934a9c8aa21d8db88f2c9e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.234869 8545 raft_consensus.cc:359] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:59.234923 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f39ed19223fa4268a01aaf4c7762d86e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f39ed19223fa4268a01aaf4c7762d86e already in progress: creating tablet
I20251028 02:59:59.235003 8545 raft_consensus.cc:385] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.235037 8545 raft_consensus.cc:740] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.235199 8545 consensus_queue.cc:260] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 02:59:59.235512 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet f39ed19223fa4268a01aaf4c7762d86e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f39ed19223fa4268a01aaf4c7762d86e already in progress: creating tablet
I20251028 02:59:59.235576 8545 ts_tablet_manager.cc:1434] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.235821 8545 tablet_bootstrap.cc:492] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.237201 8545 tablet_bootstrap.cc:654] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.238021 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet f39ed19223fa4268a01aaf4c7762d86e with delete_type TABLET_DATA_DELETED (replaced by tablet b6a2791098934a9c8aa21d8db88f2c9e) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:59.238780 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f39ed19223fa4268a01aaf4c7762d86e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f39ed19223fa4268a01aaf4c7762d86e already in progress: creating tablet
I20251028 02:59:59.239002 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet a9653fc3236f4e0b812de5a859932587 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:59.239334 8545 tablet_bootstrap.cc:492] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.239470 8545 ts_tablet_manager.cc:1403] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:59.240006 8545 raft_consensus.cc:359] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.240962 8545 raft_consensus.cc:385] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.241066 8545 raft_consensus.cc:740] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.241268 8545 consensus_queue.cc:260] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.241778 8545 ts_tablet_manager.cc:1434] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:59.242034 8545 tablet_bootstrap.cc:492] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.242478 8546 tablet_bootstrap.cc:492] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.242583 8546 ts_tablet_manager.cc:1403] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20251028 02:59:59.243109 8545 tablet_bootstrap.cc:654] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.243121 8546 raft_consensus.cc:359] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.243249 8546 raft_consensus.cc:385] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.243283 8546 raft_consensus.cc:740] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.243438 8546 consensus_queue.cc:260] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.243765 8546 ts_tablet_manager.cc:1434] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.243952 8546 tablet_bootstrap.cc:492] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.244952 8546 tablet_bootstrap.cc:654] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.246565 8546 tablet_bootstrap.cc:492] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.246711 8546 ts_tablet_manager.cc:1403] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 02:59:59.247236 8544 tablet_bootstrap.cc:492] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.247430 8544 ts_tablet_manager.cc:1403] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20251028 02:59:59.247418 8546 raft_consensus.cc:359] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.247603 8546 raft_consensus.cc:385] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.247836 8546 raft_consensus.cc:740] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.248138 8544 raft_consensus.cc:359] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.248325 8553 consensus_queue.cc:1048] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.248061 8546 consensus_queue.cc:260] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.248314 8544 raft_consensus.cc:385] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.248621 8544 raft_consensus.cc:740] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.248775 8544 consensus_queue.cc:260] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.249361 8544 ts_tablet_manager.cc:1434] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:59.249495 8546 ts_tablet_manager.cc:1434] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:59.249760 8546 tablet_bootstrap.cc:492] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.249621 8544 tablet_bootstrap.cc:492] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.250895 8546 tablet_bootstrap.cc:654] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.253675 8546 tablet_bootstrap.cc:492] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.253841 8546 ts_tablet_manager.cc:1403] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:59.254338 8546 raft_consensus.cc:359] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.254585 9401 raft_consensus.cc:493] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.254669 9401 raft_consensus.cc:515] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.255053 9401 leader_election.cc:290] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 02:59:59.255627 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "923e46aca8284966980ac5b062baf829" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:59.255985 8083 leader_election.cc:343] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.255266 8546 raft_consensus.cc:385] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.255419 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "923e46aca8284966980ac5b062baf829" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.258158 8546 raft_consensus.cc:740] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.258324 8546 consensus_queue.cc:260] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
W20251028 02:59:59.258453 8083 leader_election.cc:343] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.258518 8083 leader_election.cc:304] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:59.258920 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]). Expected version 1 got 0
I20251028 02:59:59.259080 8544 tablet_bootstrap.cc:654] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.259577 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 94cbad596db44060bac2992e9ed17ed8 (table default.Table_3af8c6d87c2c4cd1baf98aea5a596a8d [id=8dbb87b579c746e99322d3a93825d3a4]). Expected version 1 got 0
I20251028 02:59:59.260006 9401 raft_consensus.cc:2749] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:59.260326 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 3d5e016e5ae64971ab0fa0287b4a81d4 (DEFAULT_TABLE table=default.Table_2c04cfc8df68467d9ed65a7499d4e827 [id=359f27c3466f43fcbe28c1867d1c91cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.260489 8546 ts_tablet_manager.cc:1434] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20251028 02:59:59.260615 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d5e016e5ae64971ab0fa0287b4a81d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.260676 8546 tablet_bootstrap.cc:492] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.261483 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 3d5e016e5ae64971ab0fa0287b4a81d4 (DEFAULT_TABLE table=default.Table_2c04cfc8df68467d9ed65a7499d4e827 [id=359f27c3466f43fcbe28c1867d1c91cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.261596 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 73a45776f8964e68b79bc973097223a3 (table default.Table_d246152b26344aed845d6366d95b40a0 [id=a9b54811e6ba4461a372d76424f07b4f]). Expected version 2 got 0
I20251028 02:59:59.261731 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d5e016e5ae64971ab0fa0287b4a81d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.265883 8416 raft_consensus.cc:1275] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:59.266167 9595 raft_consensus.cc:493] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.266248 9595 raft_consensus.cc:515] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.266637 9595 leader_election.cc:290] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.267776 8145 raft_consensus.cc:1275] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:59.268338 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bca9ca8db1b491f9e6f7a564dc24ac1" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.268337 8553 consensus_queue.cc:1048] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.269209 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 73a45776f8964e68b79bc973097223a3 (table default.Table_d246152b26344aed845d6366d95b40a0 [id=a9b54811e6ba4461a372d76424f07b4f]). Expected version 2 got 0
I20251028 02:59:59.269304 8553 consensus_queue.cc:1048] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.269356 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]). Expected version 1 got 0
I20251028 02:59:59.269439 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bca9ca8db1b491f9e6f7a564dc24ac1" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:59.269579 8145 raft_consensus.cc:2468] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:59.271867 9840 raft_consensus.cc:493] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.271970 9840 raft_consensus.cc:515] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.274726 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 3d5e016e5ae64971ab0fa0287b4a81d4 (DEFAULT_TABLE table=default.Table_2c04cfc8df68467d9ed65a7499d4e827 [id=359f27c3466f43fcbe28c1867d1c91cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.274793 8546 tablet_bootstrap.cc:654] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.275019 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d5e016e5ae64971ab0fa0287b4a81d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.275954 8546 tablet_bootstrap.cc:492] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.276046 8546 ts_tablet_manager.cc:1403] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 02:59:59.276814 8546 raft_consensus.cc:359] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.276926 8546 raft_consensus.cc:385] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.277096 8546 raft_consensus.cc:740] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.277256 8546 consensus_queue.cc:260] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.277868 8546 ts_tablet_manager.cc:1434] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:59.278102 8546 tablet_bootstrap.cc:492] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.279690 8546 tablet_bootstrap.cc:654] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.280601 8544 tablet_bootstrap.cc:492] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
W20251028 02:59:59.280591 8220 leader_election.cc:343] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.280705 8544 ts_tablet_manager.cc:1403] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.001s
I20251028 02:59:59.280766 9840 leader_election.cc:290] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.281111 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73a45776f8964e68b79bc973097223a3" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.281242 8271 raft_consensus.cc:2468] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:59.281231 8544 raft_consensus.cc:359] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.281276 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5059e1ed0b494788bfd571987160b0b4
I20251028 02:59:59.281329 8544 raft_consensus.cc:385] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.281368 8544 raft_consensus.cc:740] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.281493 8544 consensus_queue.cc:260] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.281567 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 5059e1ed0b494788bfd571987160b0b4 deleted and replaced by tablet 5277519c4a734a75bb2ec0b380676d4b
I20251028 02:59:59.281885 8544 ts_tablet_manager.cc:1434] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:59.281919 8356 leader_election.cc:304] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:59.282106 8544 tablet_bootstrap.cc:492] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.282385 9840 raft_consensus.cc:2804] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:59.282420 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet aaaec176a3884a79a55bb74397a7dc9c with delete_type TABLET_DATA_DELETED (replaced by tablet 2bf100407b034626b4a9f7e2340f9cf0) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.282445 9840 raft_consensus.cc:493] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:59.282485 9840 raft_consensus.cc:3060] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.282554 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 4902bc3398754802a5c571036cd19fa7 with delete_type TABLET_DATA_DELETED (replaced by tablet b42f039c30a048e299c39bcf44033125) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.282738 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 5059e1ed0b494788bfd571987160b0b4 with delete_type TABLET_DATA_DELETED (replaced by tablet 5277519c4a734a75bb2ec0b380676d4b) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.282989 8551 tablet_replica.cc:333] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.283124 8544 tablet_bootstrap.cc:654] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.283128 8551 raft_consensus.cc:2243] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.283237 8551 raft_consensus.cc:2272] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.283584 9840 raft_consensus.cc:515] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.283792 8551 ts_tablet_manager.cc:1916] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.283946 9840 leader_election.cc:290] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.284736 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73a45776f8964e68b79bc973097223a3" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:59.284853 8271 raft_consensus.cc:3060] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.285844 8271 raft_consensus.cc:2468] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:59.286403 8551 ts_tablet_manager.cc:1929] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.286477 8551 log.cc:1199] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/aaaec176a3884a79a55bb74397a7dc9c
I20251028 02:59:59.286706 8551 ts_tablet_manager.cc:1950] T aaaec176a3884a79a55bb74397a7dc9c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.287403 8551 tablet_replica.cc:333] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.287792 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 5059e1ed0b494788bfd571987160b0b4 with delete_type TABLET_DATA_DELETED (replaced by tablet 5277519c4a734a75bb2ec0b380676d4b) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.287853 8551 raft_consensus.cc:2243] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.288156 8551 raft_consensus.cc:2272] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.288497 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 1de3acf65f744368ab31dc1c82ac554f (table default.Table_e5fa1e8f3825450685059a8a45433ef8 [id=739f0673774d472bbb62f176b70998bc]). Expected version 2 got 0
I20251028 02:59:59.288686 8551 ts_tablet_manager.cc:1916] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.289579 8545 tablet_bootstrap.cc:492] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.289703 8545 ts_tablet_manager.cc:1403] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.000s
I20251028 02:59:59.290376 8545 raft_consensus.cc:359] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.290529 8545 raft_consensus.cc:385] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.290591 8545 raft_consensus.cc:740] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.290773 8545 consensus_queue.cc:260] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.291358 8545 ts_tablet_manager.cc:1434] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 02:59:59.291589 8551 ts_tablet_manager.cc:1929] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.291745 8551 log.cc:1199] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4902bc3398754802a5c571036cd19fa7
I20251028 02:59:59.291981 8551 ts_tablet_manager.cc:1950] T 4902bc3398754802a5c571036cd19fa7 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.293283 8545 tablet_bootstrap.cc:492] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.293596 8416 raft_consensus.cc:1275] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:59.294008 8553 consensus_queue.cc:1048] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.294440 8544 tablet_bootstrap.cc:492] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.294588 8544 ts_tablet_manager.cc:1403] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.001s
I20251028 02:59:59.294618 8545 tablet_bootstrap.cc:654] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.295434 9839 raft_consensus.cc:493] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.295513 9839 raft_consensus.cc:515] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.295626 8356 leader_election.cc:304] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:59.295822 9190 raft_consensus.cc:2804] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:59.295882 9839 leader_election.cc:290] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.295897 9190 raft_consensus.cc:697] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:59.296033 9190 consensus_queue.cc:237] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.296911 8545 tablet_bootstrap.cc:492] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.297004 8545 ts_tablet_manager.cc:1403] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 02:59:59.297586 8545 raft_consensus.cc:359] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.297701 8545 raft_consensus.cc:385] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.297734 8545 raft_consensus.cc:740] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.297878 8545 consensus_queue.cc:260] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.298156 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 94cbad596db44060bac2992e9ed17ed8 (table default.Table_3af8c6d87c2c4cd1baf98aea5a596a8d [id=8dbb87b579c746e99322d3a93825d3a4]). Expected version 1 got 0
I20251028 02:59:59.298264 8545 ts_tablet_manager.cc:1434] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.298466 8545 tablet_bootstrap.cc:492] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.298978 8544 raft_consensus.cc:359] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.299113 8544 raft_consensus.cc:385] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.299149 8544 raft_consensus.cc:740] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.299271 8544 consensus_queue.cc:260] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.299726 8545 tablet_bootstrap.cc:654] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.299877 8544 ts_tablet_manager.cc:1434] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 02:59:59.300107 8544 tablet_bootstrap.cc:492] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 02:59:59.300293 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 5059e1ed0b494788bfd571987160b0b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5059e1ed0b494788bfd571987160b0b4 already in progress: creating tablet
I20251028 02:59:59.300433 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet aaaec176a3884a79a55bb74397a7dc9c (table default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262]) successfully deleted
I20251028 02:59:59.300534 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4902bc3398754802a5c571036cd19fa7 (table default.Table_2050be1c77ea4de0a7e510783a5f4009 [id=4a127e52978a4c8ea8a947478ce5996d]) successfully deleted
W20251028 02:59:59.300653 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 5059e1ed0b494788bfd571987160b0b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5059e1ed0b494788bfd571987160b0b4 already in progress: creating tablet
I20251028 02:59:59.301189 8544 tablet_bootstrap.cc:654] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.301714 8143 raft_consensus.cc:1275] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:59.301823 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73a45776f8964e68b79bc973097223a3" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:59.301906 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "73a45776f8964e68b79bc973097223a3" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:59.301934 8146 raft_consensus.cc:3060] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.302127 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93ef23900c0e4498934208352eb706f8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:59.302249 8142 raft_consensus.cc:2393] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7e4cf694f71438298dd32a346abcf75 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 02:59:59.302597 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "93ef23900c0e4498934208352eb706f8" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.303002 8146 raft_consensus.cc:2468] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:59.303349 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet c03faadb312742a4aecac0325f3595b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.303583 8552 tablet_replica.cc:333] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.303735 8552 raft_consensus.cc:2243] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.303913 8552 raft_consensus.cc:2272] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.304008 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 94cbad596db44060bac2992e9ed17ed8 (table default.Table_3af8c6d87c2c4cd1baf98aea5a596a8d [id=8dbb87b579c746e99322d3a93825d3a4]). Expected version 1 got 0
I20251028 02:59:59.304144 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet c21f794ec1284f5d99521954ed7ba9c9 (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]). Expected version 1 got 0
I20251028 02:59:59.304232 7968 catalog_manager.cc:5649] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:59.304414 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 73a45776f8964e68b79bc973097223a3 (table default.Table_d246152b26344aed845d6366d95b40a0 [id=a9b54811e6ba4461a372d76424f07b4f]). Expected version 2 got 0
I20251028 02:59:59.304471 8552 ts_tablet_manager.cc:1916] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.304960 8218 leader_election.cc:304] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:59.305353 9685 raft_consensus.cc:2804] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:59.305403 9685 raft_consensus.cc:493] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:59.305454 9685 raft_consensus.cc:3060] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.303747 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet a76e16d10a174adda26b8ddd49ee528c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.306411 9685 raft_consensus.cc:515] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.306803 9685 leader_election.cc:290] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.306861 8552 ts_tablet_manager.cc:1929] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.306916 8552 log.cc:1199] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c03faadb312742a4aecac0325f3595b7
I20251028 02:59:59.306890 9685 consensus_queue.cc:1048] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.307125 8552 ts_tablet_manager.cc:1950] T c03faadb312742a4aecac0325f3595b7 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.308517 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bca9ca8db1b491f9e6f7a564dc24ac1" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:59.308701 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 1f57930d5211413cb6cf4a693f7557d1 from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:59.308992 8220 leader_election.cc:343] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.309659 8354 leader_election.cc:304] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 02:59:59.310037 9639 raft_consensus.cc:3060] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.311550 8544 tablet_bootstrap.cc:492] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.311654 8544 ts_tablet_manager.cc:1403] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:59.311872 9639 raft_consensus.cc:2749] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:59.312176 8544 raft_consensus.cc:359] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.312270 8544 raft_consensus.cc:385] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.312295 8544 raft_consensus.cc:740] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.312412 8544 consensus_queue.cc:260] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.312657 9639 raft_consensus.cc:493] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.312796 8544 ts_tablet_manager.cc:1434] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:59.312752 9639 raft_consensus.cc:515] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.312968 8544 tablet_bootstrap.cc:492] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.313120 9639 leader_election.cc:290] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.314399 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8bca9ca8db1b491f9e6f7a564dc24ac1" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:59.314517 8146 raft_consensus.cc:3060] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.315675 8146 raft_consensus.cc:2468] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:59.315852 8545 tablet_bootstrap.cc:492] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.315948 8545 ts_tablet_manager.cc:1403] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.002s
I20251028 02:59:59.316601 8545 raft_consensus.cc:359] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.316723 8545 raft_consensus.cc:385] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.316777 8545 raft_consensus.cc:740] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.316915 8545 consensus_queue.cc:260] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.317286 8545 ts_tablet_manager.cc:1434] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:59.317662 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e4eec90a2fa412bb0954805f5b8d832 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.317991 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 02695b18936d4f928cd55f9b76b90b2e with delete_type TABLET_DATA_DELETED (replaced by tablet ebf449a6d241486a892eba79568c3409) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.318190 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 81c2c40e4da242f8891687a6365d273c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.318248 8545 tablet_bootstrap.cc:492] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.318463 8551 tablet_replica.cc:333] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.318589 8551 raft_consensus.cc:2243] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.318678 8551 raft_consensus.cc:2272] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.318722 8550 tablet_replica.cc:333] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.318868 8550 raft_consensus.cc:2243] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.318974 8550 raft_consensus.cc:2272] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.319212 8551 ts_tablet_manager.cc:1916] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.319386 8545 tablet_bootstrap.cc:654] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.319591 8550 ts_tablet_manager.cc:1916] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.320457 8552 tablet_replica.cc:333] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.320600 8552 raft_consensus.cc:2243] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.320701 8552 raft_consensus.cc:2272] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.321501 8544 tablet_bootstrap.cc:654] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.322158 8550 ts_tablet_manager.cc:1929] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.322239 8550 log.cc:1199] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/81c2c40e4da242f8891687a6365d273c
I20251028 02:59:59.322479 9639 raft_consensus.cc:493] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.322571 9639 raft_consensus.cc:515] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.322892 8550 ts_tablet_manager.cc:1950] T 81c2c40e4da242f8891687a6365d273c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:59.323024 9639 leader_election.cc:290] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.323280 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02695b18936d4f928cd55f9b76b90b2e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.323911 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84c3602bee3b4334a2a30daec986791d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.324029 8273 raft_consensus.cc:2468] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
W20251028 02:59:59.324188 8356 leader_election.cc:343] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.324556 8356 leader_election.cc:304] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:59.325367 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 81c2c40e4da242f8891687a6365d273c (table default.Table_5eddcfdf60ef439397a4948da2d85f90 [id=1cf0d713d3de40858825646f084e8505]) successfully deleted
I20251028 02:59:59.326102 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c03faadb312742a4aecac0325f3595b7 (table default.Table_7fa5056d9def461dab21615f8b7e8e82 [id=da02d283856342fdaba51b17ce586311]) successfully deleted
I20251028 02:59:59.326400 9639 raft_consensus.cc:2804] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:59.326467 9639 raft_consensus.cc:493] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:59.326519 9639 raft_consensus.cc:3060] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.326550 8218 leader_election.cc:304] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:59.327705 9639 raft_consensus.cc:515] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.328166 9639 leader_election.cc:290] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.328377 8553 raft_consensus.cc:2804] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:59.328462 8553 raft_consensus.cc:697] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:59.328595 8553 consensus_queue.cc:237] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.329054 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84c3602bee3b4334a2a30daec986791d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:59.329185 8273 raft_consensus.cc:3060] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.329883 9551 tablet.cc:1722] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:59.331928 8273 raft_consensus.cc:2468] T 84c3602bee3b4334a2a30daec986791d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:59.332417 8356 leader_election.cc:304] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:59.333906 8551 ts_tablet_manager.cc:1929] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.334045 8551 log.cc:1199] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/02695b18936d4f928cd55f9b76b90b2e
I20251028 02:59:59.334080 9190 raft_consensus.cc:2804] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:59.334233 9190 raft_consensus.cc:697] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:59.334439 9190 consensus_queue.cc:237] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.334671 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 5059e1ed0b494788bfd571987160b0b4 with delete_type TABLET_DATA_DELETED (replaced by tablet 5277519c4a734a75bb2ec0b380676d4b) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:59.335117 8354 leader_election.cc:343] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 02695b18936d4f928cd55f9b76b90b2e
I20251028 02:59:59.335196 8354 leader_election.cc:304] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 02:59:59.335278 8354 raft_consensus.cc:2686] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251028 02:59:59.336236 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84c3602bee3b4334a2a30daec986791d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:59.336346 8144 raft_consensus.cc:2468] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:59.336490 9529 tablet.cc:1722] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:59.336632 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84c3602bee3b4334a2a30daec986791d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:59.336731 8146 raft_consensus.cc:3060] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.337121 8551 ts_tablet_manager.cc:1950] T 02695b18936d4f928cd55f9b76b90b2e P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.337668 8146 raft_consensus.cc:2468] T 84c3602bee3b4334a2a30daec986791d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:59.341800 9401 raft_consensus.cc:493] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.341902 9401 raft_consensus.cc:515] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.342247 9401 leader_election.cc:290] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.342299 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 63ff5fe6968b47119ef0eabe691dc41c (DEFAULT_TABLE table=default.Table_a4a285cc359344a59f28244ff43d0da4 [id=8f74dbf6478042d48879ad33619c22f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.342593 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63ff5fe6968b47119ef0eabe691dc41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.342725 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13fb883843d141f09aa90e66f9b7bf1f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:59.343207 8083 leader_election.cc:343] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.343804 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 02695b18936d4f928cd55f9b76b90b2e (table default.Table_e2683e2b014947bd87e048f63aa666f1 [id=058a75d461a0497a9268f8edf3506ece]) successfully deleted
I20251028 02:59:59.344686 8398 tablet_service.cc:1505] Processing CreateTablet for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd (DEFAULT_TABLE table=default.Table_121342cd2f774cce941d56a09004daae [id=3c7fbac434d94ddd9af9f54fc83f47f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.344794 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 1fb0f525ded64eb3850ed026fac77aec (DEFAULT_TABLE table=default.Table_70cf1ece6d88419b952556cc05782215 [id=4d59fe8a5bc34a8789689bb56a112f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.344971 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.344812 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 517df20e0f3245cfb1eaf1eb52d3c6a8 (DEFAULT_TABLE table=default.Table_3da0f96b500a4ff7b76f7aaa052a52a0 [id=3a22d4d83c4a4688916cdf8c1bd4802c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.345366 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 1fb0f525ded64eb3850ed026fac77aec (DEFAULT_TABLE table=default.Table_70cf1ece6d88419b952556cc05782215 [id=4d59fe8a5bc34a8789689bb56a112f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.345517 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 517df20e0f3245cfb1eaf1eb52d3c6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.345559 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb0f525ded64eb3850ed026fac77aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.347424 9595 consensus_queue.cc:1048] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.347486 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 63ff5fe6968b47119ef0eabe691dc41c (DEFAULT_TABLE table=default.Table_a4a285cc359344a59f28244ff43d0da4 [id=8f74dbf6478042d48879ad33619c22f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.347677 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb0f525ded64eb3850ed026fac77aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.347680 8261 tablet_service.cc:1505] Processing CreateTablet for tablet e9176618f1f64e4e8ff999364bb518ee (DEFAULT_TABLE table=default.Table_a6aa4919568f4f3fbfd16ce34c062d3b [id=74cd06fb84f244d7a868b4e87230273c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.347934 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9176618f1f64e4e8ff999364bb518ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.347895 8387 tablet_service.cc:1505] Processing CreateTablet for tablet e9176618f1f64e4e8ff999364bb518ee (DEFAULT_TABLE table=default.Table_a6aa4919568f4f3fbfd16ce34c062d3b [id=74cd06fb84f244d7a868b4e87230273c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.348133 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9176618f1f64e4e8ff999364bb518ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.349049 9551 tablet.cc:1722] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.349654 9401 raft_consensus.cc:493] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.349766 9401 raft_consensus.cc:515] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.350142 9401 leader_election.cc:290] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.350262 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 63ff5fe6968b47119ef0eabe691dc41c (DEFAULT_TABLE table=default.Table_a4a285cc359344a59f28244ff43d0da4 [id=8f74dbf6478042d48879ad33619c22f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.350399 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63ff5fe6968b47119ef0eabe691dc41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.350544 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63ff5fe6968b47119ef0eabe691dc41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.352028 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5059e1ed0b494788bfd571987160b0b4" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:59.352391 8083 leader_election.cc:343] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.353076 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1f57930d5211413cb6cf4a693f7557d1
I20251028 02:59:59.353103 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 1fb0f525ded64eb3850ed026fac77aec (DEFAULT_TABLE table=default.Table_70cf1ece6d88419b952556cc05782215 [id=4d59fe8a5bc34a8789689bb56a112f27]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.353361 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb0f525ded64eb3850ed026fac77aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.353389 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 1f57930d5211413cb6cf4a693f7557d1 deleted and replaced by tablet 5e4dd3f17b7947b788b534571e549e83
I20251028 02:59:59.353509 8113 tablet_service.cc:1505] Processing CreateTablet for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd (DEFAULT_TABLE table=default.Table_121342cd2f774cce941d56a09004daae [id=3c7fbac434d94ddd9af9f54fc83f47f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.353760 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.355398 8117 tablet_service.cc:1505] Processing CreateTablet for tablet b2f9f94122d945638654639dd539244e (DEFAULT_TABLE table=default.Table_f42e5ece3b0049de9c9f81ea3d595d19 [id=b9bfe192ddd94f9e904597d9485b4077]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.355619 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2f9f94122d945638654639dd539244e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.355645 8122 tablet_service.cc:1505] Processing CreateTablet for tablet e9176618f1f64e4e8ff999364bb518ee (DEFAULT_TABLE table=default.Table_a6aa4919568f4f3fbfd16ce34c062d3b [id=74cd06fb84f244d7a868b4e87230273c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.355927 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9176618f1f64e4e8ff999364bb518ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.356216 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 5f40e16c9741411e96c430532f28158c (DEFAULT_TABLE table=default.Table_08f544277a764846bfb2cbda994300d6 [id=8f918222c0d94a9ca282ee53cce2d2d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.356403 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f40e16c9741411e96c430532f28158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.357271 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 5277519c4a734a75bb2ec0b380676d4b (DEFAULT_TABLE table=default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.357329 8263 tablet_service.cc:1505] Processing CreateTablet for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd (DEFAULT_TABLE table=default.Table_121342cd2f774cce941d56a09004daae [id=3c7fbac434d94ddd9af9f54fc83f47f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.357498 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5277519c4a734a75bb2ec0b380676d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.357581 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.357978 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 631f56367ed846308549e829b1e0ca22 (DEFAULT_TABLE table=default.Table_bb04323fd765408aa725406848f20bef [id=0432b5df96744e24b8a64627c3d49c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.358182 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 631f56367ed846308549e829b1e0ca22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.359805 8127 tablet_service.cc:1505] Processing CreateTablet for tablet b6a2791098934a9c8aa21d8db88f2c9e (DEFAULT_TABLE table=default.Table_4debdeef26b14d9b93a878210eaba4c3 [id=5a391450a3a04f01bcd5c3c06d36cf66]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.360006 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a2791098934a9c8aa21d8db88f2c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.360105 8116 tablet_service.cc:1505] Processing CreateTablet for tablet f6c75f26cb2e46be8b4e427fe9b1b076 (DEFAULT_TABLE table=default.Table_b12ebf9b73404d62b93a0eefa8d10b53 [id=4acbcdeb5d644490a70ae38d044d2c22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.360344 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6c75f26cb2e46be8b4e427fe9b1b076. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.362362 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4673cd2bd8f2439c94aac247f1ce8f12" } modify_external_catalogs: true
I20251028 02:59:59.362509 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4673cd2bd8f2439c94aac247f1ce8f12" } modify_external_catalogs: true
I20251028 02:59:59.362550 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b60ad574ddda4f7d9c66bf24dd5c612a" } modify_external_catalogs: true
I20251028 02:59:59.362617 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b60ad574ddda4f7d9c66bf24dd5c612a" } modify_external_catalogs: true
I20251028 02:59:59.362735 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13fb883843d141f09aa90e66f9b7bf1f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.362378 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d5216511b6cb4223afa3920eec378c6b" } modify_external_catalogs: true
I20251028 02:59:59.363334 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5059e1ed0b494788bfd571987160b0b4" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.363390 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d5216511b6cb4223afa3920eec378c6b" } modify_external_catalogs: true
I20251028 02:59:59.364442 8253 tablet_service.cc:1505] Processing CreateTablet for tablet b6a2791098934a9c8aa21d8db88f2c9e (DEFAULT_TABLE table=default.Table_4debdeef26b14d9b93a878210eaba4c3 [id=5a391450a3a04f01bcd5c3c06d36cf66]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.364718 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a2791098934a9c8aa21d8db88f2c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.364948 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 5f40e16c9741411e96c430532f28158c (DEFAULT_TABLE table=default.Table_08f544277a764846bfb2cbda994300d6 [id=8f918222c0d94a9ca282ee53cce2d2d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.365227 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f40e16c9741411e96c430532f28158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.365332 8257 tablet_service.cc:1505] Processing CreateTablet for tablet f6c75f26cb2e46be8b4e427fe9b1b076 (DEFAULT_TABLE table=default.Table_b12ebf9b73404d62b93a0eefa8d10b53 [id=4acbcdeb5d644490a70ae38d044d2c22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.365566 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6c75f26cb2e46be8b4e427fe9b1b076. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.367527 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 5277519c4a734a75bb2ec0b380676d4b (DEFAULT_TABLE table=default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.367810 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5277519c4a734a75bb2ec0b380676d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.368252 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 5277519c4a734a75bb2ec0b380676d4b (DEFAULT_TABLE table=default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.368391 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 5f40e16c9741411e96c430532f28158c (DEFAULT_TABLE table=default.Table_08f544277a764846bfb2cbda994300d6 [id=8f918222c0d94a9ca282ee53cce2d2d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.368494 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5277519c4a734a75bb2ec0b380676d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.368548 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f57930d5211413cb6cf4a693f7557d1 with delete_type TABLET_DATA_DELETED (replaced by tablet 5e4dd3f17b7947b788b534571e549e83) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.368624 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f40e16c9741411e96c430532f28158c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.368780 8387 tablet_service.cc:1505] Processing CreateTablet for tablet b2f9f94122d945638654639dd539244e (DEFAULT_TABLE table=default.Table_f42e5ece3b0049de9c9f81ea3d595d19 [id=b9bfe192ddd94f9e904597d9485b4077]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.368845 8384 tablet_service.cc:1505] Processing CreateTablet for tablet f6c75f26cb2e46be8b4e427fe9b1b076 (DEFAULT_TABLE table=default.Table_b12ebf9b73404d62b93a0eefa8d10b53 [id=4acbcdeb5d644490a70ae38d044d2c22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.368975 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2f9f94122d945638654639dd539244e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.369136 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6c75f26cb2e46be8b4e427fe9b1b076. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.369138 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 631f56367ed846308549e829b1e0ca22 (DEFAULT_TABLE table=default.Table_bb04323fd765408aa725406848f20bef [id=0432b5df96744e24b8a64627c3d49c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.369292 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 631f56367ed846308549e829b1e0ca22 (DEFAULT_TABLE table=default.Table_bb04323fd765408aa725406848f20bef [id=0432b5df96744e24b8a64627c3d49c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.369341 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 631f56367ed846308549e829b1e0ca22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.369482 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 631f56367ed846308549e829b1e0ca22. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.371582 8396 tablet_service.cc:1505] Processing CreateTablet for tablet b6a2791098934a9c8aa21d8db88f2c9e (DEFAULT_TABLE table=default.Table_4debdeef26b14d9b93a878210eaba4c3 [id=5a391450a3a04f01bcd5c3c06d36cf66]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.371791 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a2791098934a9c8aa21d8db88f2c9e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 02:59:59.371985 8083 leader_election.cc:343] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.372082 8083 leader_election.cc:304] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
W20251028 02:59:59.372337 8083 leader_election.cc:343] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.372390 8083 leader_election.cc:304] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:59.372897 9401 raft_consensus.cc:2749] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:59.373003 9401 raft_consensus.cc:2749] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:59.375538 7967 catalog_manager.cc:5649] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:59.376017 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f57930d5211413cb6cf4a693f7557d1 with delete_type TABLET_DATA_DELETED (replaced by tablet 5e4dd3f17b7947b788b534571e549e83) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.378624 8261 tablet_service.cc:1505] Processing CreateTablet for tablet b2f9f94122d945638654639dd539244e (DEFAULT_TABLE table=default.Table_f42e5ece3b0049de9c9f81ea3d595d19 [id=b9bfe192ddd94f9e904597d9485b4077]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.378965 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2f9f94122d945638654639dd539244e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.379817 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3872cb93268a4e86b571339ce2d7b127
I20251028 02:59:59.380560 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 59cd8e6357fe4a6cae6be04c493a6880
I20251028 02:59:59.381037 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 517df20e0f3245cfb1eaf1eb52d3c6a8 (DEFAULT_TABLE table=default.Table_3da0f96b500a4ff7b76f7aaa052a52a0 [id=3a22d4d83c4a4688916cdf8c1bd4802c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.381318 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 517df20e0f3245cfb1eaf1eb52d3c6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.383634 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 59cd8e6357fe4a6cae6be04c493a6880 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.384934 7961 catalog_manager.cc:5649] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:59.386298 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 1f57930d5211413cb6cf4a693f7557d1 with delete_type TABLET_DATA_DELETED (replaced by tablet 5e4dd3f17b7947b788b534571e549e83) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.386751 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 517df20e0f3245cfb1eaf1eb52d3c6a8 (DEFAULT_TABLE table=default.Table_3da0f96b500a4ff7b76f7aaa052a52a0 [id=3a22d4d83c4a4688916cdf8c1bd4802c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.387094 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 517df20e0f3245cfb1eaf1eb52d3c6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.387822 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 3872cb93268a4e86b571339ce2d7b127 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.388435 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 920623f3747046d9be594241e8c13733
W20251028 02:59:59.388749 7954 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 1d3e422c7ae64b25960714ecc926046a (table default.Table_e204f02c01f848fda844db72d743be8e [id=181b50c2d29a4b17b8189e3abb10a420], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1d3e422c7ae64b25960714ecc926046a consensus state has no leader
I20251028 02:59:59.390955 9809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 02:59:59.391099 9809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: This master's current role is: LEADER
I20251028 02:59:59.391708 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 920623f3747046d9be594241e8c13733 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.392943 8552 ts_tablet_manager.cc:1916] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.395493 8550 tablet_replica.cc:333] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.395732 8550 raft_consensus.cc:2243] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.395956 8550 raft_consensus.cc:2272] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.396215 8552 ts_tablet_manager.cc:1929] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.396376 8552 log.cc:1199] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/a76e16d10a174adda26b8ddd49ee528c
I20251028 02:59:59.396672 8552 ts_tablet_manager.cc:1950] T a76e16d10a174adda26b8ddd49ee528c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.396943 8550 ts_tablet_manager.cc:1916] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.397329 8552 tablet_replica.cc:333] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.397506 8552 raft_consensus.cc:2243] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.397641 8552 raft_consensus.cc:2272] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.398561 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3d306057aa1a4c9cb80d782c62f67d76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.399026 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3d306057aa1a4c9cb80d782c62f67d76 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.399539 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b7ba9c8cdfbe47f8bdb304567e48bab0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.399864 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7ba9c8cdfbe47f8bdb304567e48bab0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.400636 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2847dd26debb407b821b676dc3df8c4a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:59.400728 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 3872cb93268a4e86b571339ce2d7b127 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:59.400975 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2847dd26debb407b821b676dc3df8c4a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.401039 8552 ts_tablet_manager.cc:1916] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:59.401224 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 1f57930d5211413cb6cf4a693f7557d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f57930d5211413cb6cf4a693f7557d1 already in progress: creating tablet
I20251028 02:59:59.401647 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet 02695b18936d4f928cd55f9b76b90b2e with delete_type TABLET_DATA_DELETED (replaced by tablet ebf449a6d241486a892eba79568c3409) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.401671 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet c03faadb312742a4aecac0325f3595b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.401798 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 59cd8e6357fe4a6cae6be04c493a6880 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.401790 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet a76e16d10a174adda26b8ddd49ee528c (table default.Table_273147e41d9f4c43b46e54c10ff21e09 [id=400620abf0584fadbb666f17e41477b0]) successfully deleted
I20251028 02:59:59.401818 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 920623f3747046d9be594241e8c13733 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.401922 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 3872cb93268a4e86b571339ce2d7b127 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.402107 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_89cd288f05a043b1bfc0ac7cf4f54236"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:59.402248 8551 tablet_replica.cc:333] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.402316 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 59cd8e6357fe4a6cae6be04c493a6880 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:59.402410 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89cd288f05a043b1bfc0ac7cf4f54236 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.402411 8551 raft_consensus.cc:2243] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.402438 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 920623f3747046d9be594241e8c13733 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.402542 8551 raft_consensus.cc:2272] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.402573 8550 ts_tablet_manager.cc:1929] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.402880 8550 log.cc:1199] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/5059e1ed0b494788bfd571987160b0b4
I20251028 02:59:59.403134 8550 ts_tablet_manager.cc:1950] T 5059e1ed0b494788bfd571987160b0b4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:59.403129 8551 ts_tablet_manager.cc:1916] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.404163 8550 tablet_replica.cc:333] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.404836 8552 ts_tablet_manager.cc:1929] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.405252 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 5059e1ed0b494788bfd571987160b0b4 (table default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]) successfully deleted
W20251028 02:59:59.405418 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1f57930d5211413cb6cf4a693f7557d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f57930d5211413cb6cf4a693f7557d1 already in progress: creating tablet
W20251028 02:59:59.405558 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 59cd8e6357fe4a6cae6be04c493a6880 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59cd8e6357fe4a6cae6be04c493a6880 already in progress: creating tablet
W20251028 02:59:59.405732 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 3872cb93268a4e86b571339ce2d7b127 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3872cb93268a4e86b571339ce2d7b127 already in progress: creating tablet
I20251028 02:59:59.406571 8552 log.cc:1199] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9e4eec90a2fa412bb0954805f5b8d832
I20251028 02:59:59.406911 8552 ts_tablet_manager.cc:1950] T 9e4eec90a2fa412bb0954805f5b8d832 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.407727 8551 ts_tablet_manager.cc:1929] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.407815 8551 log.cc:1199] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/920623f3747046d9be594241e8c13733
I20251028 02:59:59.408381 8551 ts_tablet_manager.cc:1950] T 920623f3747046d9be594241e8c13733 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.408645 8552 tablet_replica.cc:333] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.408901 8552 raft_consensus.cc:2243] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:59.409122 8551 tablet_replica.cc:333] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.409258 8551 raft_consensus.cc:2243] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.409369 8551 raft_consensus.cc:2272] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.409580 8552 raft_consensus.cc:2272] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.411262 8552 ts_tablet_manager.cc:1916] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.410717 8551 ts_tablet_manager.cc:1916] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.407434 9788 raft_consensus.cc:493] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.410174 8544 tablet_bootstrap.cc:492] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.410508 9529 tablet.cc:1722] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.413439 8544 ts_tablet_manager.cc:1403] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.101s user 0.002s sys 0.001s
I20251028 02:59:59.413295 9788 raft_consensus.cc:515] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.413678 8550 raft_consensus.cc:2243] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.413785 8550 raft_consensus.cc:2272] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.413972 8544 raft_consensus.cc:359] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.414047 9788 leader_election.cc:290] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.414283 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 1979906599754f7ba049c7a9b23bf26d (table default.Table_cdab567cf8da46e8a4d7ba2a80e1e2d9 [id=9fcc70dd88724a28b404e9d4109015aa]). Expected version 1 got 0
I20251028 02:59:59.414063 8544 raft_consensus.cc:385] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.414547 8544 raft_consensus.cc:740] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.414536 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f28ca2cf41564792b49940aebc9b2c98" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.414697 8544 consensus_queue.cc:260] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.415074 8544 ts_tablet_manager.cc:1434] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20251028 02:59:59.416276 8083 leader_election.cc:343] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.416946 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f28ca2cf41564792b49940aebc9b2c98" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:59.417309 8083 leader_election.cc:343] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.417378 8083 leader_election.cc:304] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:59.417584 9788 raft_consensus.cc:2749] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:59.414477 8550 ts_tablet_manager.cc:1916] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.419108 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2f27286185dc41e4b5bc773a4ccdd68a" } new_table_name: "default.Table_fdde17a2fdc44922a634539e4c16cb75" modify_external_catalogs: true
I20251028 02:59:59.419252 8551 ts_tablet_manager.cc:1929] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.419344 8551 log.cc:1199] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c03faadb312742a4aecac0325f3595b7
I20251028 02:59:59.414414 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 1de3acf65f744368ab31dc1c82ac554f (table default.Table_e5fa1e8f3825450685059a8a45433ef8 [id=739f0673774d472bbb62f176b70998bc]). Expected version 2 got 0
I20251028 02:59:59.419456 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet c21f794ec1284f5d99521954ed7ba9c9 (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]). Expected version 1 got 0
I20251028 02:59:59.419584 8551 ts_tablet_manager.cc:1950] T c03faadb312742a4aecac0325f3595b7 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.419919 7967 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_e5fa1e8f3825450685059a8a45433ef8 [id=739f0673774d472bbb62f176b70998bc] alter complete (version 2)
I20251028 02:59:59.421452 8550 ts_tablet_manager.cc:1929] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.421583 8550 log.cc:1199] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/920623f3747046d9be594241e8c13733
I20251028 02:59:59.421911 8550 ts_tablet_manager.cc:1950] T 920623f3747046d9be594241e8c13733 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:59.423694 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 920623f3747046d9be594241e8c13733 (table default.Table_b60ad574ddda4f7d9c66bf24dd5c612a [id=bbd58ecf5bc44d18a12db6bb204c9c84]) successfully deleted
I20251028 02:59:59.423903 8552 ts_tablet_manager.cc:1929] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.424036 8552 log.cc:1199] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/920623f3747046d9be594241e8c13733
I20251028 02:59:59.424340 8552 ts_tablet_manager.cc:1950] T 920623f3747046d9be594241e8c13733 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.425338 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 81c2c40e4da242f8891687a6365d273c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:59.425755 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 81c2c40e4da242f8891687a6365d273c because the tablet was not found. No further retry: Not found: Tablet not found: 81c2c40e4da242f8891687a6365d273c
I20251028 02:59:59.427534 8544 tablet_bootstrap.cc:492] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.428793 8544 tablet_bootstrap.cc:654] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.430240 8544 tablet_bootstrap.cc:492] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.430328 8544 ts_tablet_manager.cc:1403] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20251028 02:59:59.433506 8544 raft_consensus.cc:359] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.433736 8544 raft_consensus.cc:385] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.433836 8544 raft_consensus.cc:740] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.434060 8544 consensus_queue.cc:260] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.434533 8544 ts_tablet_manager.cc:1434] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:59.434818 8544 tablet_bootstrap.cc:492] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.435918 8544 tablet_bootstrap.cc:654] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.438398 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92] alter complete (version 1)
I20251028 02:59:59.439446 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bc8cc7f063644f3eaa9fb0abce91091a" } new_table_name: "default.Table_c5aa8eae0ee64a58abe614181b524864" modify_external_catalogs: true
I20251028 02:59:59.440635 8273 raft_consensus.cc:1275] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:59.441080 9840 consensus_queue.cc:1048] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.444093 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a19c30e391a34b00816d39f3426a855d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.444480 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a19c30e391a34b00816d39f3426a855d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.447250 9888 raft_consensus.cc:493] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.447429 9888 raft_consensus.cc:515] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.449077 7960 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 73a45776f8964e68b79bc973097223a3 (table default.Table_d246152b26344aed845d6366d95b40a0 [id=a9b54811e6ba4461a372d76424f07b4f]). Expected version 2 got 0
I20251028 02:59:59.449205 9888 leader_election.cc:290] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.450395 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e95a45cb612447dd9bebd9f7a019d1f9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:59.450798 8083 leader_election.cc:343] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.451188 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e95a45cb612447dd9bebd9f7a019d1f9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:59.451594 8083 leader_election.cc:343] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.451661 8083 leader_election.cc:304] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:59.451792 9888 raft_consensus.cc:2749] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:59.452387 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2af68266ea4d4f288ecc63e064bb69db" } modify_external_catalogs: true
I20251028 02:59:59.452498 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2af68266ea4d4f288ecc63e064bb69db" } modify_external_catalogs: true
I20251028 02:59:59.453912 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a2f9a31f1859418e9dca1b761f5025e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.454389 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2f9a31f1859418e9dca1b761f5025e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.456141 8544 tablet_bootstrap.cc:492] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.456259 8544 ts_tablet_manager.cc:1403] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20251028 02:59:59.456899 8544 raft_consensus.cc:359] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.456931 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 5e4dd3f17b7947b788b534571e549e83 (DEFAULT_TABLE table=default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.457028 8544 raft_consensus.cc:385] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.457065 8544 raft_consensus.cc:740] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.457239 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e4dd3f17b7947b788b534571e549e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.457192 8544 consensus_queue.cc:260] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.457597 8544 ts_tablet_manager.cc:1434] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 02:59:59.457851 8544 tablet_bootstrap.cc:492] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.459180 8544 tablet_bootstrap.cc:654] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.460951 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet b4ef27777bc047b886a1be0d4622dcf8
I20251028 02:59:59.462880 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet b4ef27777bc047b886a1be0d4622dcf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.463380 9885 consensus_queue.cc:1048] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:59.463907 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet b4ef27777bc047b886a1be0d4622dcf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b4ef27777bc047b886a1be0d4622dcf8 already in progress: creating tablet
I20251028 02:59:59.468720 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_9c2247ee8bfa4320b2c634f44cddbee9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.469201 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c2247ee8bfa4320b2c634f44cddbee9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.471025 8146 raft_consensus.cc:1275] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:59.472726 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e204f02c01f848fda844db72d743be8e" } new_table_name: "default.Table_d9b8ba18e24848b1b8511a66b8c7cf2f" modify_external_catalogs: true
I20251028 02:59:59.472895 9888 raft_consensus.cc:493] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.473001 9888 raft_consensus.cc:515] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.473416 9888 leader_election.cc:290] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.474139 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1979906599754f7ba049c7a9b23bf26d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.474376 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1979906599754f7ba049c7a9b23bf26d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:59.474625 8083 leader_election.cc:343] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.475662 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_94da0c3c823c4016b6534ddecabf64a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.476056 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94da0c3c823c4016b6534ddecabf64a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.477191 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_08d693721d15425a85eb515a90b30096"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.477512 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08d693721d15425a85eb515a90b30096 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.478152 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6cad41d6014141378b584a34027d2caf" } new_table_name: "default.Table_8161374f7dab4a769d9fcfe2e982ae99" modify_external_catalogs: true
I20251028 02:59:59.478230 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0ed734dfd7104e248da86872fcab5120"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.478518 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ed734dfd7104e248da86872fcab5120 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.479178 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8314d35830564f179d7e69e756fefa45" } new_table_name: "default.Table_c8fb6d535ce34fbe8032195cf9e43f57" modify_external_catalogs: true
I20251028 02:59:59.481458 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 5e4dd3f17b7947b788b534571e549e83 (DEFAULT_TABLE table=default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.481810 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e4dd3f17b7947b788b534571e549e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.482375 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet b4ef27777bc047b886a1be0d4622dcf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.482913 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9e4eec90a2fa412bb0954805f5b8d832 (table default.Table_90d5c8bb11484e5ab7648f8cb5d5c4e0 [id=a4f9578fb0c146a7af9769f101d57da5]) successfully deleted
W20251028 02:59:59.483124 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 1f57930d5211413cb6cf4a693f7557d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f57930d5211413cb6cf4a693f7557d1 already in progress: creating tablet
W20251028 02:59:59.483387 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 3872cb93268a4e86b571339ce2d7b127 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3872cb93268a4e86b571339ce2d7b127 already in progress: creating tablet
I20251028 02:59:59.485318 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet b4ef27777bc047b886a1be0d4622dcf8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.485642 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 5e4dd3f17b7947b788b534571e549e83 (DEFAULT_TABLE table=default.Table_632504e5f4754d1582584291b0702d3e [id=be26076bec294678bf7a4d2d56a97a7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.485921 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e4dd3f17b7947b788b534571e549e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.487622 9733 tablet.cc:1722] T fef84b69b1b746d4b4feeed96670c67b P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:59.488371 9551 tablet.cc:1722] T fef84b69b1b746d4b4feeed96670c67b P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:59.488422 9840 consensus_queue.cc:1048] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:59.492957 8083 leader_election.cc:343] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.493052 8083 leader_election.cc:304] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:59.493719 9888 raft_consensus.cc:2749] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251028 02:59:59.497210 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 59cd8e6357fe4a6cae6be04c493a6880 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59cd8e6357fe4a6cae6be04c493a6880 already in progress: creating tablet
I20251028 02:59:59.497349 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 920623f3747046d9be594241e8c13733 (table default.Table_b60ad574ddda4f7d9c66bf24dd5c612a [id=bbd58ecf5bc44d18a12db6bb204c9c84]) successfully deleted
W20251028 02:59:59.497493 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet b4ef27777bc047b886a1be0d4622dcf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b4ef27777bc047b886a1be0d4622dcf8 already in progress: creating tablet
W20251028 02:59:59.497716 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 02695b18936d4f928cd55f9b76b90b2e because the tablet was not found. No further retry: Not found: Tablet not found: 02695b18936d4f928cd55f9b76b90b2e
W20251028 02:59:59.497833 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 59cd8e6357fe4a6cae6be04c493a6880 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59cd8e6357fe4a6cae6be04c493a6880 already in progress: creating tablet
I20251028 02:59:59.497918 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 920623f3747046d9be594241e8c13733 (table default.Table_b60ad574ddda4f7d9c66bf24dd5c612a [id=bbd58ecf5bc44d18a12db6bb204c9c84]) successfully deleted
I20251028 02:59:59.498005 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet c03faadb312742a4aecac0325f3595b7 (table default.Table_7fa5056d9def461dab21615f8b7e8e82 [id=da02d283856342fdaba51b17ce586311]) successfully deleted
W20251028 02:59:59.498095 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 3872cb93268a4e86b571339ce2d7b127 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3872cb93268a4e86b571339ce2d7b127 already in progress: creating tablet
W20251028 02:59:59.498256 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet b4ef27777bc047b886a1be0d4622dcf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b4ef27777bc047b886a1be0d4622dcf8 already in progress: creating tablet
W20251028 02:59:59.487874 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5fa32a1433cd47a9ad39d1c613a18761 consensus state has no leader
W20251028 02:59:59.503649 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1d3e422c7ae64b25960714ecc926046a consensus state has no leader
I20251028 02:59:59.505002 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet 322cbd8253ce45118ad14e8f8f2f1f74 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:59.505795 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet f92acda752d5463aae30ce7e63f2f16f from {username='slave'} at 127.0.0.1:47364
W20251028 02:59:59.507177 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 338cf9d02142436f97ccd44313cc8199 consensus state has no leader
I20251028 02:59:59.507377 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:59.492434 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7ceb18d43d6c41d2b7809c8c97a5f2e7" } modify_external_catalogs: true
I20251028 02:59:59.508373 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7ceb18d43d6c41d2b7809c8c97a5f2e7" } modify_external_catalogs: true
I20251028 02:59:59.511082 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 900973d832c24b62a4554ae9a219cfd8 (table default.Table_f0dafbe2e0a84541a79cf6c887d59619 [id=c4aadbebe43f451ea22467feaa6dafbb]). Expected version 1 got 0
I20251028 02:59:59.511440 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 09d65eef81b2421c92678015b6da4fbb (table default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1]). Expected version 1 got 0
I20251028 02:59:59.514070 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 6bc54972260747f3af0c18d05c6a3442 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:59.520226 9888 raft_consensus.cc:493] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.520341 9888 raft_consensus.cc:515] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.520902 8421 raft_consensus.cc:1275] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:59.521310 9888 leader_election.cc:290] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.521581 9685 consensus_queue.cc:1048] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.523175 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94cbad596db44060bac2992e9ed17ed8" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 02:59:59.523576 8083 leader_election.cc:343] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.524065 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94cbad596db44060bac2992e9ed17ed8" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.524211 8421 raft_consensus.cc:2468] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 02:59:59.525725 8083 leader_election.cc:304] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 02:59:59.525919 9888 raft_consensus.cc:2804] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:59.525974 9888 raft_consensus.cc:493] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:59.526021 9888 raft_consensus.cc:3060] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.527094 9888 raft_consensus.cc:515] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.527498 9888 leader_election.cc:290] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.528147 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 322cbd8253ce45118ad14e8f8f2f1f74
I20251028 02:59:59.528460 7962 catalog_manager.cc:6719] ReplaceTablet: tablet 322cbd8253ce45118ad14e8f8f2f1f74 deleted and replaced by tablet 95f4e2921c97438ab9db2878d85cf66e
I20251028 02:59:59.528434 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94cbad596db44060bac2992e9ed17ed8" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:59.528872 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "94cbad596db44060bac2992e9ed17ed8" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:59.528995 8421 raft_consensus.cc:3060] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 02:59:59.530164 8083 leader_election.cc:343] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.530221 8421 raft_consensus.cc:2468] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 02:59:59.530428 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 322cbd8253ce45118ad14e8f8f2f1f74 with delete_type TABLET_DATA_DELETED (replaced by tablet 95f4e2921c97438ab9db2878d85cf66e) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.530598 8550 tablet_replica.cc:333] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.530730 8550 raft_consensus.cc:2243] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.530850 8550 raft_consensus.cc:2272] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.530845 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 322cbd8253ce45118ad14e8f8f2f1f74 with delete_type TABLET_DATA_DELETED (replaced by tablet 95f4e2921c97438ab9db2878d85cf66e) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.531035 8551 tablet_replica.cc:333] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.531184 8551 raft_consensus.cc:2243] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:59.531407 8551 raft_consensus.cc:2272] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.531520 8550 ts_tablet_manager.cc:1916] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.531637 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 322cbd8253ce45118ad14e8f8f2f1f74 with delete_type TABLET_DATA_DELETED (replaced by tablet 95f4e2921c97438ab9db2878d85cf66e) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.532123 8083 leader_election.cc:304] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 02:59:59.532207 8552 tablet_replica.cc:333] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.532373 8552 raft_consensus.cc:2243] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.532487 8552 raft_consensus.cc:2272] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.533089 8552 ts_tablet_manager.cc:1916] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.533917 9788 raft_consensus.cc:2804] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:59.533977 8550 ts_tablet_manager.cc:1929] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.534006 9788 raft_consensus.cc:697] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 02:59:59.534055 8550 log.cc:1199] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/322cbd8253ce45118ad14e8f8f2f1f74
I20251028 02:59:59.534153 9788 consensus_queue.cc:237] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.534315 8550 ts_tablet_manager.cc:1950] T 322cbd8253ce45118ad14e8f8f2f1f74 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:59.534982 9733 tablet.cc:1722] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:59.535523 8145 raft_consensus.cc:1275] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:59.536517 8551 ts_tablet_manager.cc:1916] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.537293 9733 tablet.cc:1722] T 369397a89baa472285b25acd6c78e7ec P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.537292 9551 tablet.cc:1722] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.537743 8553 raft_consensus.cc:493] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.537837 8553 raft_consensus.cc:515] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.538195 8553 leader_election.cc:290] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.538750 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f6f58ddaa6940248065a05b75996112" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.538885 8420 raft_consensus.cc:2468] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:59.539105 7967 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_fdde17a2fdc44922a634539e4c16cb75 [id=cf715387170e40c992807874e34c66b7] alter complete (version 2)
I20251028 02:59:59.539399 8220 leader_election.cc:304] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:59.540169 9595 raft_consensus.cc:2804] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:59.540223 9595 raft_consensus.cc:493] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:59.540266 9595 raft_consensus.cc:3060] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.541294 9595 raft_consensus.cc:515] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.541631 9595 leader_election.cc:290] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.543139 7962 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 900973d832c24b62a4554ae9a219cfd8 (table default.Table_f0dafbe2e0a84541a79cf6c887d59619 [id=c4aadbebe43f451ea22467feaa6dafbb]). Expected version 1 got 0
I20251028 02:59:59.538801 8551 ts_tablet_manager.cc:1929] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.543712 8551 log.cc:1199] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/322cbd8253ce45118ad14e8f8f2f1f74
I20251028 02:59:59.543763 8552 ts_tablet_manager.cc:1929] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.543864 8552 log.cc:1199] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/322cbd8253ce45118ad14e8f8f2f1f74
I20251028 02:59:59.544000 8551 ts_tablet_manager.cc:1950] T 322cbd8253ce45118ad14e8f8f2f1f74 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.544162 8552 ts_tablet_manager.cc:1950] T 322cbd8253ce45118ad14e8f8f2f1f74 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.544492 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f6f58ddaa6940248065a05b75996112" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 02:59:59.544600 8420 raft_consensus.cc:3060] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.545171 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 322cbd8253ce45118ad14e8f8f2f1f74 (table default.Table_610f127f3f9348309aa1ce4b6537e5b1 [id=40f8798a68d54ea991ef9bf6887dfe36]) successfully deleted
I20251028 02:59:59.545383 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 322cbd8253ce45118ad14e8f8f2f1f74 (table default.Table_610f127f3f9348309aa1ce4b6537e5b1 [id=40f8798a68d54ea991ef9bf6887dfe36]) successfully deleted
I20251028 02:59:59.545596 8420 raft_consensus.cc:2468] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:59.546046 8220 leader_election.cc:304] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:59.546635 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 322cbd8253ce45118ad14e8f8f2f1f74 (table default.Table_610f127f3f9348309aa1ce4b6537e5b1 [id=40f8798a68d54ea991ef9bf6887dfe36]) successfully deleted
I20251028 02:59:59.550056 9906 tablet.cc:1722] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.550274 9595 raft_consensus.cc:2804] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:59.550370 9595 raft_consensus.cc:697] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 02:59:59.550493 9595 consensus_queue.cc:237] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.558656 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_d246152b26344aed845d6366d95b40a0 [id=a9b54811e6ba4461a372d76424f07b4f] alter complete (version 2)
I20251028 02:59:59.558976 8553 consensus_queue.cc:1048] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.561554 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f6f58ddaa6940248065a05b75996112" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 02:59:59.561697 8146 raft_consensus.cc:2468] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 02:59:59.564802 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_f0dafbe2e0a84541a79cf6c887d59619 [id=c4aadbebe43f451ea22467feaa6dafbb] alter complete (version 1)
I20251028 02:59:59.570676 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f6f58ddaa6940248065a05b75996112" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 02:59:59.570855 8146 raft_consensus.cc:3060] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.572072 8146 raft_consensus.cc:2468] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 02:59:59.573551 9906 tablet.cc:1722] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:59.573765 9529 tablet.cc:1722] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.581516 9190 consensus_queue.cc:1048] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.583719 9909 tablet.cc:1722] T 369397a89baa472285b25acd6c78e7ec P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.587795 9910 tablet.cc:1722] T fef84b69b1b746d4b4feeed96670c67b P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:59.596487 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_8161374f7dab4a769d9fcfe2e982ae99 [id=8c5f437ae7374d68b232dbb3e43cc8aa] alter complete (version 1)
I20251028 02:59:59.598888 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_66f629fcdad8471bb69ea0c30537b139" } modify_external_catalogs: true
I20251028 02:59:59.599056 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_66f629fcdad8471bb69ea0c30537b139" } modify_external_catalogs: true
I20251028 02:59:59.598914 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_18181568bf2c4744a84690b6dac5f13f" } modify_external_catalogs: true
I20251028 02:59:59.600106 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_18181568bf2c4744a84690b6dac5f13f" } modify_external_catalogs: true
I20251028 02:59:59.601627 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]). Expected version 2 got 0
I20251028 02:59:59.601761 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet c21f794ec1284f5d99521954ed7ba9c9 (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]). Expected version 1 got 0
I20251028 02:59:59.603645 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_01d864857be34675882e1be7ba5a9a7c" } modify_external_catalogs: true
I20251028 02:59:59.604007 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_01d864857be34675882e1be7ba5a9a7c" } modify_external_catalogs: true
I20251028 02:59:59.604804 9733 tablet.cc:1722] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.604868 9551 tablet.cc:1722] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:59.606338 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5428db004de74f7492c9b8999e53020e
I20251028 02:59:59.607131 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 5428db004de74f7492c9b8999e53020e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.607331 8550 tablet_replica.cc:333] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.607491 8550 raft_consensus.cc:2243] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.607617 8550 raft_consensus.cc:2272] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.608052 8550 ts_tablet_manager.cc:1916] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.608934 9906 tablet.cc:1722] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:59.611003 8550 ts_tablet_manager.cc:1929] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:59.611089 8550 log.cc:1199] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/5428db004de74f7492c9b8999e53020e
I20251028 02:59:59.615047 7962 catalog_manager.cc:5649] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 02:59:59.615255 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]). Expected version 1 got 0
I20251028 02:59:59.617794 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 5428db004de74f7492c9b8999e53020e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.618063 8552 tablet_replica.cc:333] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.618264 8552 raft_consensus.cc:2243] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:59.618307 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 5428db004de74f7492c9b8999e53020e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.618496 8552 raft_consensus.cc:2272] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.618865 8551 tablet_replica.cc:333] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.619005 8552 ts_tablet_manager.cc:1916] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.619036 8551 raft_consensus.cc:2243] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.619158 8551 raft_consensus.cc:2272] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.619562 8551 ts_tablet_manager.cc:1916] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.621503 9551 tablet.cc:1722] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.621894 8552 ts_tablet_manager.cc:1929] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:59.621958 8552 log.cc:1199] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/5428db004de74f7492c9b8999e53020e
I20251028 02:59:59.621953 8551 ts_tablet_manager.cc:1929] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:59.622027 8551 log.cc:1199] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/5428db004de74f7492c9b8999e53020e
I20251028 02:59:59.622314 8551 ts_tablet_manager.cc:1950] T 5428db004de74f7492c9b8999e53020e P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.622854 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 55e72a7f8aa842049da7aeaaa7001a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.622201 8552 ts_tablet_manager.cc:1950] T 5428db004de74f7492c9b8999e53020e P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.624399 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]) successfully deleted
I20251028 02:59:59.625082 8387 tablet_service.cc:1505] Processing CreateTablet for tablet ad8b00751973482494b4fc0a7cfdf6f0 (DEFAULT_TABLE table=default.Table_89cd288f05a043b1bfc0ac7cf4f54236 [id=3a1b28cf59444632873c2350c7a9c3b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.625363 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad8b00751973482494b4fc0a7cfdf6f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.625422 8113 tablet_service.cc:1505] Processing CreateTablet for tablet ad8b00751973482494b4fc0a7cfdf6f0 (DEFAULT_TABLE table=default.Table_89cd288f05a043b1bfc0ac7cf4f54236 [id=3a1b28cf59444632873c2350c7a9c3b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.625671 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad8b00751973482494b4fc0a7cfdf6f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.626034 7967 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee] alter complete (version 2)
I20251028 02:59:59.629439 8251 tablet_service.cc:1505] Processing CreateTablet for tablet ad8b00751973482494b4fc0a7cfdf6f0 (DEFAULT_TABLE table=default.Table_89cd288f05a043b1bfc0ac7cf4f54236 [id=3a1b28cf59444632873c2350c7a9c3b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.629644 8550 ts_tablet_manager.cc:1950] T 5428db004de74f7492c9b8999e53020e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:59.629748 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad8b00751973482494b4fc0a7cfdf6f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.630085 8550 tablet_replica.cc:333] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.630169 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]) successfully deleted
I20251028 02:59:59.630218 8550 raft_consensus.cc:2243] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.630311 8550 raft_consensus.cc:2272] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.630808 8550 ts_tablet_manager.cc:1916] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.632174 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6bc54972260747f3af0c18d05c6a3442
I20251028 02:59:59.632462 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 6bc54972260747f3af0c18d05c6a3442 deleted and replaced by tablet 9d438110cfec42a991b6c40b0881cc62
I20251028 02:59:59.632651 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bc54972260747f3af0c18d05c6a3442 with delete_type TABLET_DATA_DELETED (replaced by tablet 9d438110cfec42a991b6c40b0881cc62) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.633188 8550 ts_tablet_manager.cc:1929] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.633262 8550 log.cc:1199] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/55e72a7f8aa842049da7aeaaa7001a28
I20251028 02:59:59.633265 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bc54972260747f3af0c18d05c6a3442 with delete_type TABLET_DATA_DELETED (replaced by tablet 9d438110cfec42a991b6c40b0881cc62) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.633491 8550 ts_tablet_manager.cc:1950] T 55e72a7f8aa842049da7aeaaa7001a28 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 02:59:59.633728 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 6bc54972260747f3af0c18d05c6a3442 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6bc54972260747f3af0c18d05c6a3442 already in progress: creating tablet
I20251028 02:59:59.633929 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 55e72a7f8aa842049da7aeaaa7001a28 (table default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]) successfully deleted
I20251028 02:59:59.634107 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bc54972260747f3af0c18d05c6a3442 with delete_type TABLET_DATA_DELETED (replaced by tablet 9d438110cfec42a991b6c40b0881cc62) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.635342 7967 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 73a45776f8964e68b79bc973097223a3 (table default.Table_d246152b26344aed845d6366d95b40a0 [id=a9b54811e6ba4461a372d76424f07b4f]). Expected version 2 got 0
I20251028 02:59:59.635901 7786 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
/tmp/dist-test-taskJrsfcH/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.190:39091
--webserver_interface=127.7.154.190
--webserver_port=40487
--builtin_ntp_servers=127.7.154.148:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20251028 02:59:59.637748 9401 raft_consensus.cc:493] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.637853 9401 raft_consensus.cc:515] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.638242 9401 leader_election.cc:290] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.638860 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6bd72bb34fe843079f8a4b828770e441" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 02:59:59.639055 9910 tablet.cc:1722] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20251028 02:59:59.639261 8083 leader_election.cc:343] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.639956 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet af23fe1145a54b76a9f8e7249b87ca0d
I20251028 02:59:59.641019 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6bd72bb34fe843079f8a4b828770e441" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.641971 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet af23fe1145a54b76a9f8e7249b87ca0d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.642131 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet af23fe1145a54b76a9f8e7249b87ca0d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.642302 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet af23fe1145a54b76a9f8e7249b87ca0d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:59.642940 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet af23fe1145a54b76a9f8e7249b87ca0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af23fe1145a54b76a9f8e7249b87ca0d already in progress: creating tablet
W20251028 02:59:59.643734 8083 leader_election.cc:343] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.643848 8083 leader_election.cc:304] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:59.644019 9401 raft_consensus.cc:2749] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:59.645174 9733 tablet.cc:1722] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.645524 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f92acda752d5463aae30ce7e63f2f16f
I20251028 02:59:59.645836 7963 catalog_manager.cc:6719] ReplaceTablet: tablet f92acda752d5463aae30ce7e63f2f16f deleted and replaced by tablet d5b660efe0bc4feea0c09383ad2aa682
I20251028 02:59:59.646445 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet f92acda752d5463aae30ce7e63f2f16f with delete_type TABLET_DATA_DELETED (replaced by tablet d5b660efe0bc4feea0c09383ad2aa682) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:59.646883 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f92acda752d5463aae30ce7e63f2f16f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f92acda752d5463aae30ce7e63f2f16f already in progress: creating tablet
I20251028 02:59:59.647084 7967 catalog_manager.cc:5649] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } }
I20251028 02:59:59.647239 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet f92acda752d5463aae30ce7e63f2f16f with delete_type TABLET_DATA_DELETED (replaced by tablet d5b660efe0bc4feea0c09383ad2aa682) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.647411 7967 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 94cbad596db44060bac2992e9ed17ed8 (table default.Table_3af8c6d87c2c4cd1baf98aea5a596a8d [id=8dbb87b579c746e99322d3a93825d3a4]). Expected version 1 got 0
I20251028 02:59:59.648399 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 5428db004de74f7492c9b8999e53020e (table default.Table_66f629fcdad8471bb69ea0c30537b139 [id=d40c0ee824ba423395e475f68276b5d7]) successfully deleted
W20251028 02:59:59.649865 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 6bc54972260747f3af0c18d05c6a3442 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6bc54972260747f3af0c18d05c6a3442 already in progress: creating tablet
W20251028 02:59:59.650009 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet af23fe1145a54b76a9f8e7249b87ca0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af23fe1145a54b76a9f8e7249b87ca0d already in progress: creating tablet
W20251028 02:59:59.650126 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet f92acda752d5463aae30ce7e63f2f16f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f92acda752d5463aae30ce7e63f2f16f already in progress: creating tablet
I20251028 02:59:59.652691 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet f92acda752d5463aae30ce7e63f2f16f with delete_type TABLET_DATA_DELETED (replaced by tablet d5b660efe0bc4feea0c09383ad2aa682) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.655337 9401 raft_consensus.cc:493] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.655447 9401 raft_consensus.cc:515] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.655879 9401 leader_election.cc:290] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 02:59:59.656270 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b436561f0bf44e0b2c680ce30919318" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.655776 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 12eecf12cac641eaad5a9b5af5890941
I20251028 02:59:59.656553 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b436561f0bf44e0b2c680ce30919318" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 02:59:59.656666 8083 leader_election.cc:343] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 02:59:59.656952 8083 leader_election.cc:343] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 02:59:59.657027 8083 leader_election.cc:304] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 02:59:59.657297 9401 raft_consensus.cc:2749] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 02:59:59.657794 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 12eecf12cac641eaad5a9b5af5890941 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.658795 7968 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_594adc002945439ba87c71677fcb4ad1 [id=6d4246a1b1ea4448b8fbf6a5abecfed1] alter complete (version 1)
I20251028 02:59:59.658885 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 12eecf12cac641eaad5a9b5af5890941 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:59.659050 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 12eecf12cac641eaad5a9b5af5890941 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12eecf12cac641eaad5a9b5af5890941 already in progress: creating tablet
I20251028 02:59:59.659247 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8e202e53053248a7a9cee6a217999489
W20251028 02:59:59.659823 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 12eecf12cac641eaad5a9b5af5890941 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12eecf12cac641eaad5a9b5af5890941 already in progress: creating tablet
W20251028 02:59:59.659874 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6bc54972260747f3af0c18d05c6a3442 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6bc54972260747f3af0c18d05c6a3442 already in progress: creating tablet
W20251028 02:59:59.659973 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet af23fe1145a54b76a9f8e7249b87ca0d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af23fe1145a54b76a9f8e7249b87ca0d already in progress: creating tablet
I20251028 02:59:59.659976 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 8e202e53053248a7a9cee6a217999489 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:59.660054 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f92acda752d5463aae30ce7e63f2f16f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f92acda752d5463aae30ce7e63f2f16f already in progress: creating tablet
I20251028 02:59:59.660101 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 12eecf12cac641eaad5a9b5af5890941 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:59.660394 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 8e202e53053248a7a9cee6a217999489 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e202e53053248a7a9cee6a217999489 already in progress: creating tablet
I20251028 02:59:59.660573 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 8e202e53053248a7a9cee6a217999489 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 02:59:59.660725 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 12eecf12cac641eaad5a9b5af5890941 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12eecf12cac641eaad5a9b5af5890941 already in progress: creating tablet
I20251028 02:59:59.661159 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 8e202e53053248a7a9cee6a217999489 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:59.661357 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 8e202e53053248a7a9cee6a217999489 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e202e53053248a7a9cee6a217999489 already in progress: creating tablet
W20251028 02:59:59.661676 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 8e202e53053248a7a9cee6a217999489 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e202e53053248a7a9cee6a217999489 already in progress: creating tablet
I20251028 02:59:59.664151 9551 tablet.cc:1722] T 369397a89baa472285b25acd6c78e7ec P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.666513 8649 mvcc.cc:204] Tried to move back new op lower bound from 7215597156104925184 to 7215597155517702144. Current Snapshot: MvccSnapshot[applied={T|T < 7215597156104925184}]
I20251028 02:59:59.667555 9551 tablet.cc:1722] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:59.669431 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 73a45776f8964e68b79bc973097223a3 (table default.Table_d246152b26344aed845d6366d95b40a0 [id=a9b54811e6ba4461a372d76424f07b4f]). Expected version 2 got 0
I20251028 02:59:59.672291 9839 consensus_queue.cc:1048] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.675209 9910 tablet.cc:1722] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.675783 9652 mvcc.cc:204] Tried to move back new op lower bound from 7215597156753690624 to 7215597154830807040. Current Snapshot: MvccSnapshot[applied={T|T < 7215597156753690624}]
I20251028 02:59:59.675884 9909 tablet.cc:1722] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 02:59:59.677940 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 3a04270a2fbd49c294c4a2f24672036d (DEFAULT_TABLE table=default.Table_2847dd26debb407b821b676dc3df8c4a [id=489f88418aae4976a45ac6af74246636]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.678272 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a04270a2fbd49c294c4a2f24672036d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.678457 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 3a04270a2fbd49c294c4a2f24672036d (DEFAULT_TABLE table=default.Table_2847dd26debb407b821b676dc3df8c4a [id=489f88418aae4976a45ac6af74246636]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.678735 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a04270a2fbd49c294c4a2f24672036d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.681188 8545 tablet_bootstrap.cc:492] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.681288 8545 ts_tablet_manager.cc:1403] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.363s user 0.000s sys 0.002s
I20251028 02:59:59.681814 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 0164e659f3cd49b6b1ac6d1e8d2b4ab8 (DEFAULT_TABLE table=default.Table_b7ba9c8cdfbe47f8bdb304567e48bab0 [id=82b22b888fbc451cb4440af49ebd86b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.682073 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0164e659f3cd49b6b1ac6d1e8d2b4ab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.682994 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a96f9bde439b4ee0a079bb871e47b3a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:59.683125 8544 tablet_bootstrap.cc:492] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.683220 8544 ts_tablet_manager.cc:1403] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.225s user 0.002s sys 0.000s
I20251028 02:59:59.681836 8545 raft_consensus.cc:359] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
W20251028 02:59:59.683432 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a96f9bde439b4ee0a079bb871e47b3a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.683482 8545 raft_consensus.cc:385] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.683516 8545 raft_consensus.cc:740] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.683663 8545 consensus_queue.cc:260] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.683765 8544 raft_consensus.cc:359] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.683874 8544 raft_consensus.cc:385] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.683907 8544 raft_consensus.cc:740] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.683960 8545 ts_tablet_manager.cc:1434] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251028 02:59:59.684032 8544 consensus_queue.cc:260] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.684180 8545 tablet_bootstrap.cc:492] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.684340 8544 ts_tablet_manager.cc:1434] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.684499 8544 tablet_bootstrap.cc:492] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.684491 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_008b1b0c028d4fdface45a3b2110508d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.684850 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_008b1b0c028d4fdface45a3b2110508d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.685245 8545 tablet_bootstrap.cc:654] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.685531 8544 tablet_bootstrap.cc:654] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.685621 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_173b483053f948f58173aadecf7e6765"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.685935 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_173b483053f948f58173aadecf7e6765 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.686316 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 0164e659f3cd49b6b1ac6d1e8d2b4ab8 (DEFAULT_TABLE table=default.Table_b7ba9c8cdfbe47f8bdb304567e48bab0 [id=82b22b888fbc451cb4440af49ebd86b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.686616 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0164e659f3cd49b6b1ac6d1e8d2b4ab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.687402 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_28717f0c640748cca2efe4a69c6b3a2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.687701 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28717f0c640748cca2efe4a69c6b3a2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.687904 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b6dbdbfa2618496c8858dabf12f02366"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:59.688158 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 0696349affc34573b08bcb84df29c201 (DEFAULT_TABLE table=default.Table_3d306057aa1a4c9cb80d782c62f67d76 [id=f0896adb6fdd4e5ca97035727d601d5c]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 02:59:59.688225 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b6dbdbfa2618496c8858dabf12f02366 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.688397 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0696349affc34573b08bcb84df29c201. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.688460 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d28a01f0c50b410eb906392280a7144c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.688726 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d28a01f0c50b410eb906392280a7144c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.688894 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_feec1045abaf4ffb9ef0e698d450a0c7" } modify_external_catalogs: true
I20251028 02:59:59.691227 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 0696349affc34573b08bcb84df29c201 (DEFAULT_TABLE table=default.Table_3d306057aa1a4c9cb80d782c62f67d76 [id=f0896adb6fdd4e5ca97035727d601d5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.691506 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0696349affc34573b08bcb84df29c201. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.691511 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 3a04270a2fbd49c294c4a2f24672036d (DEFAULT_TABLE table=default.Table_2847dd26debb407b821b676dc3df8c4a [id=489f88418aae4976a45ac6af74246636]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.691864 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a04270a2fbd49c294c4a2f24672036d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.692317 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 0696349affc34573b08bcb84df29c201 (DEFAULT_TABLE table=default.Table_3d306057aa1a4c9cb80d782c62f67d76 [id=f0896adb6fdd4e5ca97035727d601d5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.692569 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0696349affc34573b08bcb84df29c201. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.691227 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 0164e659f3cd49b6b1ac6d1e8d2b4ab8 (DEFAULT_TABLE table=default.Table_b7ba9c8cdfbe47f8bdb304567e48bab0 [id=82b22b888fbc451cb4440af49ebd86b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 02:59:59.694166 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0164e659f3cd49b6b1ac6d1e8d2b4ab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 02:59:59.699465 9595 consensus_queue.cc:1048] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 02:59:59.701138 8083 consensus_peers.cc:597] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 02:59:59.701460 8419 raft_consensus.cc:1275] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 02:59:59.701854 9401 consensus_queue.cc:1048] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.702369 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_610f127f3f9348309aa1ce4b6537e5b1" } new_table_name: "default.Table_c158a2a1e7d14570a486cdbbc6044efd" modify_external_catalogs: true
I20251028 02:59:59.704260 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 93ef23900c0e4498934208352eb706f8 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:59.719036 8545 tablet_bootstrap.cc:492] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.719166 8545 ts_tablet_manager.cc:1403] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.001s
I20251028 02:59:59.719772 8545 raft_consensus.cc:359] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.719897 8545 raft_consensus.cc:385] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.719937 8545 raft_consensus.cc:740] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.720072 8545 consensus_queue.cc:260] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.720387 8545 ts_tablet_manager.cc:1434] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.720603 8545 tablet_bootstrap.cc:492] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.721711 8545 tablet_bootstrap.cc:654] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.723476 9884 consensus_queue.cc:1048] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.728368 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_feec1045abaf4ffb9ef0e698d450a0c7" } modify_external_catalogs: true
I20251028 02:59:59.730358 8546 tablet_bootstrap.cc:492] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.730463 8546 ts_tablet_manager.cc:1403] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.452s user 0.000s sys 0.002s
I20251028 02:59:59.730927 8546 raft_consensus.cc:359] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.731020 8546 raft_consensus.cc:385] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.731047 8546 raft_consensus.cc:740] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.731161 8546 consensus_queue.cc:260] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.731444 8546 ts_tablet_manager.cc:1434] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:59.731630 8546 tablet_bootstrap.cc:492] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.731899 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet b993932fd567416984ad6d28f8cadad6 with delete_type TABLET_DATA_DELETED (replaced by tablet 5c1994484abf4b6fbbc3f47527558cdb) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.732601 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1e1f686b7dda4c03b402b11bae923b8e" } new_table_name: "default.Table_49f2ad3b1c934eaeab556927604ddf1b" modify_external_catalogs: true
I20251028 02:59:59.732729 8546 tablet_bootstrap.cc:654] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.733461 8551 tablet_replica.cc:333] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.733610 8551 raft_consensus.cc:2243] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.733718 8551 raft_consensus.cc:2272] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.734026 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4675220a671f4306b8f46597ab3ca721" } new_table_name: "default.Table_d6d4af0e22af416ca63e8c2fda795eef" modify_external_catalogs: true
I20251028 02:59:59.735999 8551 ts_tablet_manager.cc:1916] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.738250 8551 ts_tablet_manager.cc:1929] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.738333 8551 log.cc:1199] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/b993932fd567416984ad6d28f8cadad6
I20251028 02:59:59.738544 8551 ts_tablet_manager.cc:1950] T b993932fd567416984ad6d28f8cadad6 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.739105 8545 tablet_bootstrap.cc:492] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.739194 8545 ts_tablet_manager.cc:1403] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20251028 02:59:59.739259 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet b993932fd567416984ad6d28f8cadad6 (table default.Table_2e728b4bb7394afaa5a9fd93d7f2739e [id=b0407c8f78a84245a954c67e888da3ec]) successfully deleted
I20251028 02:59:59.739722 8545 raft_consensus.cc:359] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.739827 8545 raft_consensus.cc:385] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.739868 8545 raft_consensus.cc:740] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.739989 8545 consensus_queue.cc:260] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.742213 8545 ts_tablet_manager.cc:1434] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:59.742364 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 94cbad596db44060bac2992e9ed17ed8 (table default.Table_3af8c6d87c2c4cd1baf98aea5a596a8d [id=8dbb87b579c746e99322d3a93825d3a4]). Expected version 1 got 0
I20251028 02:59:59.743191 8545 tablet_bootstrap.cc:492] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.745663 8545 tablet_bootstrap.cc:654] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.746915 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet a76e16d10a174adda26b8ddd49ee528c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.750221 8551 tablet_replica.cc:333] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.750396 8551 raft_consensus.cc:2243] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.750543 8551 raft_consensus.cc:2272] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.751050 8551 ts_tablet_manager.cc:1916] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.754231 8551 ts_tablet_manager.cc:1929] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.754400 8551 log.cc:1199] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a76e16d10a174adda26b8ddd49ee528c
I20251028 02:59:59.754902 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet d39008ac6324451a9d1ed54a2ccc3abf with delete_type TABLET_DATA_DELETED (replaced by tablet 985a0daa0a544216bbb06b1bc9d19231) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.755163 8551 ts_tablet_manager.cc:1950] T a76e16d10a174adda26b8ddd49ee528c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.755479 8550 tablet_replica.cc:333] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.755688 8550 raft_consensus.cc:2243] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.755837 8550 raft_consensus.cc:2272] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.756083 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a76e16d10a174adda26b8ddd49ee528c (table default.Table_273147e41d9f4c43b46e54c10ff21e09 [id=400620abf0584fadbb666f17e41477b0]) successfully deleted
I20251028 02:59:59.757990 8550 ts_tablet_manager.cc:1916] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.759500 8545 tablet_bootstrap.cc:492] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.759604 8545 ts_tablet_manager.cc:1403] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 02:59:59.760136 8545 raft_consensus.cc:359] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.760255 8545 raft_consensus.cc:385] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.760286 8545 raft_consensus.cc:740] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.760422 8545 consensus_queue.cc:260] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.760540 8550 ts_tablet_manager.cc:1929] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.760607 8550 log.cc:1199] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/d39008ac6324451a9d1ed54a2ccc3abf
I20251028 02:59:59.763119 8545 ts_tablet_manager.cc:1434] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:59.763387 9909 tablet.cc:1722] T 94cbad596db44060bac2992e9ed17ed8 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 02:59:59.763664 8545 tablet_bootstrap.cc:492] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.764822 8545 tablet_bootstrap.cc:654] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.774348 8419 raft_consensus.cc:3060] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.776552 9685 consensus_queue.cc:1048] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.778833 8545 tablet_bootstrap.cc:492] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.778935 8545 ts_tablet_manager.cc:1403] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 02:59:59.779734 8545 raft_consensus.cc:359] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.779839 8545 raft_consensus.cc:385] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.779870 8545 raft_consensus.cc:740] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.779984 8545 consensus_queue.cc:260] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.780287 8545 ts_tablet_manager.cc:1434] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.780272 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 07509e5c36d9440fa9fe29c251931c63
I20251028 02:59:59.780493 8545 tablet_bootstrap.cc:492] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.780547 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 94cbad596db44060bac2992e9ed17ed8 (table default.Table_3af8c6d87c2c4cd1baf98aea5a596a8d [id=8dbb87b579c746e99322d3a93825d3a4]). Expected version 1 got 0
I20251028 02:59:59.780769 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5eb75de4388641e4bffe66784f304fcc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.781229 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5eb75de4388641e4bffe66784f304fcc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.781311 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0ed734dfd7104e248da86872fcab5120" } new_table_name: "default.Table_e4376f0772b24edbb1b92e585184ff59" modify_external_catalogs: true
I20251028 02:59:59.781596 8545 tablet_bootstrap.cc:654] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 02:59:59.782218 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 69e0358b593f4b55b6b4ec859e2c021d consensus state has no leader
I20251028 02:59:59.782435 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_861351bbb646467db2388b94e51e0d7e" } new_table_name: "default.Table_4fe6d656356041dcaad42a9fd6f026d5" modify_external_catalogs: true
I20251028 02:59:59.783092 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 07509e5c36d9440fa9fe29c251931c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.783245 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 9e4eec90a2fa412bb0954805f5b8d832 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.783408 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 07509e5c36d9440fa9fe29c251931c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 02:59:59.783620 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1a3132020baa4d9a83a76df511fac5c6 consensus state has no leader
I20251028 02:59:59.783852 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e7879514239f4bf68e2e60480543af6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.783969 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 07509e5c36d9440fa9fe29c251931c63 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07509e5c36d9440fa9fe29c251931c63 already in progress: creating tablet
W20251028 02:59:59.784255 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7879514239f4bf68e2e60480543af6f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.787915 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_cb0c959c105d4fbc94aed34c2d82ef44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.788305 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb0c959c105d4fbc94aed34c2d82ef44 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.788450 9839 raft_consensus.cc:493] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 02:59:59.788548 9839 raft_consensus.cc:515] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.788909 9839 leader_election.cc:290] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.788458 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 5059e1ed0b494788bfd571987160b0b4 with delete_type TABLET_DATA_DELETED (replaced by tablet 5277519c4a734a75bb2ec0b380676d4b) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.790403 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5059e1ed0b494788bfd571987160b0b4" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 02:59:59.790535 8273 raft_consensus.cc:2468] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 02:59:59.791064 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5b8939a4349f4b168b7fb7a1003050b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.791451 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b8939a4349f4b168b7fb7a1003050b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.792199 8356 leader_election.cc:304] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:59.792657 9839 raft_consensus.cc:2804] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 02:59:59.792716 9839 raft_consensus.cc:493] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 02:59:59.792761 9839 raft_consensus.cc:3060] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.793818 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 93ef23900c0e4498934208352eb706f8
I20251028 02:59:59.793797 9839 raft_consensus.cc:515] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.794072 7967 catalog_manager.cc:6719] ReplaceTablet: tablet 93ef23900c0e4498934208352eb706f8 deleted and replaced by tablet 7e7d519bd243449199f83dc0219169b9
I20251028 02:59:59.794159 9839 leader_election.cc:290] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 02:59:59.794266 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 302e0ed455854eb2b661afc2f2b6921d from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:59.794967 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5059e1ed0b494788bfd571987160b0b4" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 02:59:59.795047 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 93ef23900c0e4498934208352eb706f8 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e7d519bd243449199f83dc0219169b9) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.795084 8273 raft_consensus.cc:3060] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 02:59:59.795231 8552 tablet_replica.cc:333] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.795399 8552 raft_consensus.cc:2243] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:59.795447 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 93ef23900c0e4498934208352eb706f8 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e7d519bd243449199f83dc0219169b9) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.795603 8552 raft_consensus.cc:2272] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.796294 8552 ts_tablet_manager.cc:1916] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.796557 8273 raft_consensus.cc:2468] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 02:59:59.796838 8356 leader_election.cc:304] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 02:59:59.798280 9839 raft_consensus.cc:2804] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 02:59:59.798364 9839 raft_consensus.cc:697] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 02:59:59.798486 9839 consensus_queue.cc:237] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.798682 8552 ts_tablet_manager.cc:1929] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.798751 8552 log.cc:1199] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/93ef23900c0e4498934208352eb706f8
I20251028 02:59:59.799024 8552 ts_tablet_manager.cc:1950] T 93ef23900c0e4498934208352eb706f8 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.799270 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 07509e5c36d9440fa9fe29c251931c63 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.799674 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 93ef23900c0e4498934208352eb706f8 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e7d519bd243449199f83dc0219169b9) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.802297 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 93ef23900c0e4498934208352eb706f8 (table default.Table_00a06d6bfdff457ab18a0404c944f780 [id=27cec05ebbf74601b28ed49f36637dbb]) successfully deleted
I20251028 02:59:59.804049 8544 tablet_bootstrap.cc:492] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.804165 8544 ts_tablet_manager.cc:1403] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.120s user 0.002s sys 0.000s
I20251028 02:59:59.804741 8544 raft_consensus.cc:359] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.804884 8544 raft_consensus.cc:385] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.804922 8544 raft_consensus.cc:740] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.805085 8544 consensus_queue.cc:260] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.805449 8544 ts_tablet_manager.cc:1434] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.805655 8544 tablet_bootstrap.cc:492] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.808648 8544 tablet_bootstrap.cc:654] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.811793 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 5059e1ed0b494788bfd571987160b0b4 with delete_type TABLET_DATA_DELETED (replaced by tablet 5277519c4a734a75bb2ec0b380676d4b) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.812062 8552 tablet_replica.cc:333] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.812227 8552 raft_consensus.cc:2243] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.812332 8552 raft_consensus.cc:2272] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.812619 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 94cbad596db44060bac2992e9ed17ed8 (table default.Table_3af8c6d87c2c4cd1baf98aea5a596a8d [id=8dbb87b579c746e99322d3a93825d3a4]). Expected version 1 got 0
I20251028 02:59:59.813722 9906 tablet.cc:1722] T 94cbad596db44060bac2992e9ed17ed8 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251028 02:59:59.814495 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 07509e5c36d9440fa9fe29c251931c63 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07509e5c36d9440fa9fe29c251931c63 already in progress: creating tablet
I20251028 02:59:59.814725 8551 tablet_replica.cc:333] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.814879 8551 raft_consensus.cc:2243] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.814997 8551 raft_consensus.cc:2272] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.815546 8552 ts_tablet_manager.cc:1916] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.815721 8551 ts_tablet_manager.cc:1916] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.818040 8552 ts_tablet_manager.cc:1929] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.818127 8552 log.cc:1199] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/5059e1ed0b494788bfd571987160b0b4
I20251028 02:59:59.818336 8552 ts_tablet_manager.cc:1950] T 5059e1ed0b494788bfd571987160b0b4 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.818984 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 5059e1ed0b494788bfd571987160b0b4 (table default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]) successfully deleted
I20251028 02:59:59.819495 8551 ts_tablet_manager.cc:1929] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.820341 8551 log.cc:1199] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9e4eec90a2fa412bb0954805f5b8d832
I20251028 02:59:59.823207 8551 ts_tablet_manager.cc:1950] T 9e4eec90a2fa412bb0954805f5b8d832 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 02:59:59.823827 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 95f4e2921c97438ab9db2878d85cf66e has no consensus state
I20251028 02:59:59.824016 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9e4eec90a2fa412bb0954805f5b8d832 (table default.Table_90d5c8bb11484e5ab7648f8cb5d5c4e0 [id=a4f9578fb0c146a7af9769f101d57da5]) successfully deleted
I20251028 02:59:59.824435 8551 tablet_replica.cc:333] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.824615 8551 raft_consensus.cc:2243] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:59.824817 8551 pending_rounds.cc:70] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 02:59:59.824890 8551 pending_rounds.cc:77] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597157575884800 op_type: NO_OP noop_request { }
I20251028 02:59:59.825012 8551 raft_consensus.cc:2889] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 02:59:59.825115 8551 raft_consensus.cc:2272] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.828648 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_74d9969b0d494560bbaa47fec5f802d9" } modify_external_catalogs: true
W20251028 02:59:59.833599 8354 leader_election.cc:343] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 5059e1ed0b494788bfd571987160b0b4
W20251028 02:59:59.833936 8354 leader_election.cc:343] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 5059e1ed0b494788bfd571987160b0b4
I20251028 02:59:59.835248 8550 ts_tablet_manager.cc:1950] T d39008ac6324451a9d1ed54a2ccc3abf P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:59.835240 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_74d9969b0d494560bbaa47fec5f802d9" } modify_external_catalogs: true
I20251028 02:59:59.835083 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 46ae536d269d4cbdbcee44a26d25bd67 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:59.835776 8550 tablet_replica.cc:333] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.835927 8550 raft_consensus.cc:2243] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.835906 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet d39008ac6324451a9d1ed54a2ccc3abf (table default.Table_f3795491c9e547af98943428398101b2 [id=8fe996a5bf644b1a9a0d2d3894445a56]) successfully deleted
I20251028 02:59:59.836042 8550 raft_consensus.cc:2272] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 02:59:59.836215 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 07509e5c36d9440fa9fe29c251931c63 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07509e5c36d9440fa9fe29c251931c63 already in progress: creating tablet
I20251028 02:59:59.836680 8550 ts_tablet_manager.cc:1916] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 02:59:59.838480 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9d438110cfec42a991b6c40b0881cc62 has no consensus state
I20251028 02:59:59.839275 8546 tablet_bootstrap.cc:492] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.839404 8546 ts_tablet_manager.cc:1403] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.108s user 0.002s sys 0.001s
I20251028 02:59:59.839890 8546 raft_consensus.cc:359] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.839990 8546 raft_consensus.cc:385] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.840019 8546 raft_consensus.cc:740] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.840133 8546 consensus_queue.cc:260] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.840381 8546 ts_tablet_manager.cc:1434] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:59.840584 8546 tablet_bootstrap.cc:492] T 94cbad596db44060bac2992e9ed17ed8 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.845292 8546 tablet_bootstrap.cc:654] T 94cbad596db44060bac2992e9ed17ed8 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.845845 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5002cbe7bf3044e4baaa8e884eb9283f" } modify_external_catalogs: true
I20251028 02:59:59.845970 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5002cbe7bf3044e4baaa8e884eb9283f" } modify_external_catalogs: true
I20251028 02:59:59.847374 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet b993932fd567416984ad6d28f8cadad6 with delete_type TABLET_DATA_DELETED (replaced by tablet 5c1994484abf4b6fbbc3f47527558cdb) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.851382 8552 tablet_replica.cc:333] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.851552 8552 raft_consensus.cc:2243] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.851665 8552 raft_consensus.cc:2272] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.852092 8552 ts_tablet_manager.cc:1916] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.855662 8552 ts_tablet_manager.cc:1929] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.855773 8552 log.cc:1199] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/b993932fd567416984ad6d28f8cadad6
I20251028 02:59:59.855990 8552 ts_tablet_manager.cc:1950] T b993932fd567416984ad6d28f8cadad6 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.856691 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet b993932fd567416984ad6d28f8cadad6 (table default.Table_2e728b4bb7394afaa5a9fd93d7f2739e [id=b0407c8f78a84245a954c67e888da3ec]) successfully deleted
W20251028 02:59:59.858098 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab07d25dbdca4ce583e1fa9cbd5f9d47 has no consensus state
I20251028 02:59:59.859787 7967 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_3af8c6d87c2c4cd1baf98aea5a596a8d [id=8dbb87b579c746e99322d3a93825d3a4] alter complete (version 1)
I20251028 02:59:59.860937 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7778bd93ae56485f921a20d0df78a3ec" } modify_external_catalogs: true
I20251028 02:59:59.860913 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6eabdbe1000e4fb6b82c9425cc2e0edd" } modify_external_catalogs: true
I20251028 02:59:59.861116 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6eabdbe1000e4fb6b82c9425cc2e0edd" } modify_external_catalogs: true
I20251028 02:59:59.861451 9639 consensus_queue.cc:1048] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.861104 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7778bd93ae56485f921a20d0df78a3ec" } modify_external_catalogs: true
I20251028 02:59:59.866886 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 302e0ed455854eb2b661afc2f2b6921d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 02:59:59.869549 9926 consensus_queue.cc:1175] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d7e4cf694f71438298dd32a346abcf75" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20251028 02:59:59.871498 8551 ts_tablet_manager.cc:1916] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.872869 8544 tablet_bootstrap.cc:492] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.872967 8544 ts_tablet_manager.cc:1403] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.067s user 0.002s sys 0.001s
I20251028 02:59:59.873462 8544 raft_consensus.cc:359] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.873570 8544 raft_consensus.cc:385] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.873602 8544 raft_consensus.cc:740] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.873708 8544 consensus_queue.cc:260] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.873975 8544 ts_tablet_manager.cc:1434] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.874009 8551 ts_tablet_manager.cc:1929] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.874082 8551 log.cc:1199] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/5059e1ed0b494788bfd571987160b0b4
I20251028 02:59:59.874154 8544 tablet_bootstrap.cc:492] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.874351 8551 ts_tablet_manager.cc:1950] T 5059e1ed0b494788bfd571987160b0b4 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.875192 8544 tablet_bootstrap.cc:654] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.876893 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 5059e1ed0b494788bfd571987160b0b4 (table default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]) successfully deleted
I20251028 02:59:59.877286 9639 consensus_queue.cc:1048] T 84c3602bee3b4334a2a30daec986791d P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.877887 8551 tablet_replica.cc:333] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.878093 8551 raft_consensus.cc:2243] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.878265 8551 raft_consensus.cc:2272] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.879630 8551 ts_tablet_manager.cc:1916] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.882174 8551 ts_tablet_manager.cc:1929] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.882313 8551 log.cc:1199] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/93ef23900c0e4498934208352eb706f8
I20251028 02:59:59.882635 8551 ts_tablet_manager.cc:1950] T 93ef23900c0e4498934208352eb706f8 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.883365 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 93ef23900c0e4498934208352eb706f8 (table default.Table_00a06d6bfdff457ab18a0404c944f780 [id=27cec05ebbf74601b28ed49f36637dbb]) successfully deleted
I20251028 02:59:59.886076 8550 ts_tablet_manager.cc:1929] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 02:59:59.886900 8545 tablet_bootstrap.cc:492] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.886991 8545 ts_tablet_manager.cc:1403] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.107s user 0.002s sys 0.000s
I20251028 02:59:59.887526 8545 raft_consensus.cc:359] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.887629 8545 raft_consensus.cc:385] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.887660 8545 raft_consensus.cc:740] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.887789 8545 consensus_queue.cc:260] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.887850 8546 tablet_bootstrap.cc:492] T 94cbad596db44060bac2992e9ed17ed8 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.887933 8546 ts_tablet_manager.cc:1403] T 94cbad596db44060bac2992e9ed17ed8 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.047s user 0.000s sys 0.002s
I20251028 02:59:59.888123 8545 ts_tablet_manager.cc:1434] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:59.888319 8545 tablet_bootstrap.cc:492] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.888442 8546 raft_consensus.cc:359] T 94cbad596db44060bac2992e9ed17ed8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.888532 8546 raft_consensus.cc:385] T 94cbad596db44060bac2992e9ed17ed8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.888561 8546 raft_consensus.cc:740] T 94cbad596db44060bac2992e9ed17ed8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.888680 8546 consensus_queue.cc:260] T 94cbad596db44060bac2992e9ed17ed8 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.891891 8546 ts_tablet_manager.cc:1434] T 94cbad596db44060bac2992e9ed17ed8 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:59.889520 8545 tablet_bootstrap.cc:654] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.892532 8553 consensus_queue.cc:1048] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.894271 8546 tablet_bootstrap.cc:492] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 02:59:59.896492 8546 tablet_bootstrap.cc:654] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.897241 8550 log.cc:1199] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/93ef23900c0e4498934208352eb706f8
I20251028 02:59:59.897548 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f04c867f0ce4ab99f980cca063abc30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.898133 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 94cbad596db44060bac2992e9ed17ed8 (table default.Table_3af8c6d87c2c4cd1baf98aea5a596a8d [id=8dbb87b579c746e99322d3a93825d3a4]). Expected version 1 got 0
I20251028 02:59:59.899024 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 5fa32a1433cd47a9ad39d1c613a18761 from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:59.900421 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet 5277519c4a734a75bb2ec0b380676d4b from {username='slave'} at 127.0.0.1:47364
I20251028 02:59:59.900919 8551 tablet_replica.cc:333] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.901068 8551 raft_consensus.cc:2243] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.901170 8551 raft_consensus.cc:2272] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.901183 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 94cbad596db44060bac2992e9ed17ed8 (table default.Table_3af8c6d87c2c4cd1baf98aea5a596a8d [id=8dbb87b579c746e99322d3a93825d3a4]). Expected version 1 got 0
I20251028 02:59:59.905177 8550 ts_tablet_manager.cc:1950] T 93ef23900c0e4498934208352eb706f8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:59.905277 8551 ts_tablet_manager.cc:1916] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.905920 8550 tablet_replica.cc:333] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.906721 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 93ef23900c0e4498934208352eb706f8 (table default.Table_00a06d6bfdff457ab18a0404c944f780 [id=27cec05ebbf74601b28ed49f36637dbb]) successfully deleted
I20251028 02:59:59.907720 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f13374a1b255403e9df82157903a9a0e
I20251028 02:59:59.908375 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet f13374a1b255403e9df82157903a9a0e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.909884 8550 raft_consensus.cc:2243] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.910032 8550 raft_consensus.cc:2272] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.910152 8546 tablet_bootstrap.cc:492] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 02:59:59.910306 8546 ts_tablet_manager.cc:1403] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20251028 02:59:59.910704 8550 ts_tablet_manager.cc:1916] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.910699 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet f13374a1b255403e9df82157903a9a0e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.910907 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet f13374a1b255403e9df82157903a9a0e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.910895 8546 raft_consensus.cc:359] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.911029 8546 raft_consensus.cc:385] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20251028 02:59:59.912019 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet f13374a1b255403e9df82157903a9a0e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f13374a1b255403e9df82157903a9a0e already in progress: creating tablet
I20251028 02:59:59.912550 8551 ts_tablet_manager.cc:1929] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.912684 8551 log.cc:1199] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3f04c867f0ce4ab99f980cca063abc30
I20251028 02:59:59.912961 8551 ts_tablet_manager.cc:1950] T 3f04c867f0ce4ab99f980cca063abc30 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.911067 8546 raft_consensus.cc:740] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.913210 8546 consensus_queue.cc:260] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.913764 8546 ts_tablet_manager.cc:1434] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251028 02:59:59.913812 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3f04c867f0ce4ab99f980cca063abc30 (table default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]) successfully deleted
I20251028 02:59:59.913966 8546 tablet_bootstrap.cc:492] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 02:59:59.914330 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f13374a1b255403e9df82157903a9a0e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f13374a1b255403e9df82157903a9a0e already in progress: creating tablet
I20251028 02:59:59.915390 8545 tablet_bootstrap.cc:492] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.915472 8545 ts_tablet_manager.cc:1403] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.002s
I20251028 02:59:59.915910 8545 raft_consensus.cc:359] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.916008 8545 raft_consensus.cc:385] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.916038 8545 raft_consensus.cc:740] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.916145 8545 consensus_queue.cc:260] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.916383 8545 ts_tablet_manager.cc:1434] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.916536 8545 tablet_bootstrap.cc:492] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.917196 8546 tablet_bootstrap.cc:654] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.917529 8545 tablet_bootstrap.cc:654] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.923785 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 02695b18936d4f928cd55f9b76b90b2e with delete_type TABLET_DATA_DELETED (replaced by tablet ebf449a6d241486a892eba79568c3409) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.923987 8552 tablet_replica.cc:333] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.924134 8552 raft_consensus.cc:2243] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.924252 8552 raft_consensus.cc:2272] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.924783 8552 ts_tablet_manager.cc:1916] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.926538 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 0e20b1f36d3f4f70a93ad8a0b29d6687
I20251028 02:59:59.927354 8552 ts_tablet_manager.cc:1929] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.927490 8552 log.cc:1199] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/02695b18936d4f928cd55f9b76b90b2e
I20251028 02:59:59.927515 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.927708 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.927794 8552 ts_tablet_manager.cc:1950] T 02695b18936d4f928cd55f9b76b90b2e P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.927959 8551 tablet_replica.cc:333] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.928112 8551 raft_consensus.cc:2243] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.928236 8551 raft_consensus.cc:2272] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.928244 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.928501 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 02695b18936d4f928cd55f9b76b90b2e (table default.Table_e2683e2b014947bd87e048f63aa666f1 [id=058a75d461a0497a9268f8edf3506ece]) successfully deleted
I20251028 02:59:59.928606 8551 ts_tablet_manager.cc:1916] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.928807 8552 tablet_replica.cc:333] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 02:59:59.928930 8552 raft_consensus.cc:2243] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 02:59:59.929105 8552 raft_consensus.cc:2272] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.929471 8552 ts_tablet_manager.cc:1916] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.932441 8552 ts_tablet_manager.cc:1929] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:59.932518 8552 log.cc:1199] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/0e20b1f36d3f4f70a93ad8a0b29d6687
I20251028 02:59:59.932778 8552 ts_tablet_manager.cc:1950] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 02:59:59.932911 8545 tablet_bootstrap.cc:492] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.933009 8545 ts_tablet_manager.cc:1403] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 02:59:59.933463 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 (table default.Table_7778bd93ae56485f921a20d0df78a3ec [id=7a2a6ab3c9744942a9413c6cb057f527]) successfully deleted
I20251028 02:59:59.933502 8550 ts_tablet_manager.cc:1929] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.933473 8545 raft_consensus.cc:359] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.933568 8550 log.cc:1199] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/302e0ed455854eb2b661afc2f2b6921d
I20251028 02:59:59.933574 8545 raft_consensus.cc:385] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.933609 8545 raft_consensus.cc:740] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.933777 8550 ts_tablet_manager.cc:1950] T 302e0ed455854eb2b661afc2f2b6921d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 02:59:59.933740 8545 consensus_queue.cc:260] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.934837 8544 tablet_bootstrap.cc:492] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.934923 8544 ts_tablet_manager.cc:1403] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.000s
I20251028 02:59:59.935427 8544 raft_consensus.cc:359] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.935526 8544 raft_consensus.cc:385] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.935554 8544 raft_consensus.cc:740] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.935659 8544 consensus_queue.cc:260] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.935911 8544 ts_tablet_manager.cc:1434] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.936090 8544 tablet_bootstrap.cc:492] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.936223 8545 ts_tablet_manager.cc:1434] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:59.936400 8545 tablet_bootstrap.cc:492] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.937253 8544 tablet_bootstrap.cc:654] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.938362 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_77bf3f2df56847a5988614b150aa5831"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.938778 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_77bf3f2df56847a5988614b150aa5831 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.939757 8545 tablet_bootstrap.cc:654] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.939721 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b7c2ed1bcbef411dbf047fbd023915b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.940058 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7c2ed1bcbef411dbf047fbd023915b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.940434 8551 ts_tablet_manager.cc:1929] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 02:59:59.940502 8551 log.cc:1199] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/0e20b1f36d3f4f70a93ad8a0b29d6687
I20251028 02:59:59.940768 8551 ts_tablet_manager.cc:1950] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.940905 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_55aa729834954e5cadb191355a8a754e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 02:59:59.941288 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55aa729834954e5cadb191355a8a754e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.947664 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 (table default.Table_7778bd93ae56485f921a20d0df78a3ec [id=7a2a6ab3c9744942a9413c6cb057f527]) successfully deleted
I20251028 02:59:59.948014 8545 tablet_bootstrap.cc:492] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.948172 8545 ts_tablet_manager.cc:1403] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:59.950117 8545 raft_consensus.cc:359] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.950325 8545 raft_consensus.cc:385] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.950390 8545 raft_consensus.cc:740] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.950580 8545 consensus_queue.cc:260] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 02:59:59.952574 8544 tablet_bootstrap.cc:492] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.952740 8544 ts_tablet_manager.cc:1403] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 02:59:59.954375 8545 ts_tablet_manager.cc:1434] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251028 02:59:59.955390 8544 raft_consensus.cc:359] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.955595 8544 raft_consensus.cc:385] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.955682 8544 raft_consensus.cc:740] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.955886 8544 consensus_queue.cc:260] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.956388 8545 tablet_bootstrap.cc:492] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.957634 8544 ts_tablet_manager.cc:1434] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 02:59:59.957892 8544 tablet_bootstrap.cc:492] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.959441 8545 tablet_bootstrap.cc:654] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.960623 8544 tablet_bootstrap.cc:654] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.960695 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 302e0ed455854eb2b661afc2f2b6921d (table default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]) successfully deleted
W20251028 02:59:59.961359 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f13374a1b255403e9df82157903a9a0e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f13374a1b255403e9df82157903a9a0e already in progress: creating tablet
I20251028 02:59:59.961819 8550 tablet_replica.cc:333] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 02:59:59.964848 8545 tablet_bootstrap.cc:492] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.964999 8545 ts_tablet_manager.cc:1403] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20251028 02:59:59.965492 8545 raft_consensus.cc:359] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.965662 8545 raft_consensus.cc:385] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.965672 8550 raft_consensus.cc:2243] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.965729 8545 raft_consensus.cc:740] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.965924 8545 consensus_queue.cc:260] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.967633 8545 ts_tablet_manager.cc:1434] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251028 02:59:59.967981 8545 tablet_bootstrap.cc:492] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.966900 8544 tablet_bootstrap.cc:492] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 02:59:59.970052 8545 tablet_bootstrap.cc:654] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.971140 8545 tablet_bootstrap.cc:492] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.971215 8545 ts_tablet_manager.cc:1403] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20251028 02:59:59.969193 8550 raft_consensus.cc:2272] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.970227 8544 ts_tablet_manager.cc:1403] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 02:59:59.971673 8545 raft_consensus.cc:359] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.971773 8545 raft_consensus.cc:385] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.971805 8545 raft_consensus.cc:740] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.971920 8545 consensus_queue.cc:260] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.971975 8544 raft_consensus.cc:359] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.972072 8544 raft_consensus.cc:385] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.972103 8544 raft_consensus.cc:740] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.972223 8545 ts_tablet_manager.cc:1434] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 02:59:59.972229 8544 consensus_queue.cc:260] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 02:59:59.972409 8545 tablet_bootstrap.cc:492] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.973321 8550 ts_tablet_manager.cc:1916] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.973444 8545 tablet_bootstrap.cc:654] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.974946 7965 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet b9b5fb938357499b8888ba29d23612bd (table default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]). Expected version 1 got 0
I20251028 02:59:59.976040 8544 ts_tablet_manager.cc:1434] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 02:59:59.976366 8544 tablet_bootstrap.cc:492] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 02:59:59.977453 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 with delete_type TABLET_DATA_DELETED (replaced by tablet d37c267aaee7412b869753164556b9c0) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.977653 8544 tablet_bootstrap.cc:654] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.979586 8545 tablet_bootstrap.cc:492] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.979681 8545 ts_tablet_manager.cc:1403] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20251028 02:59:59.980194 8545 raft_consensus.cc:359] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.980212 8553 consensus_queue.cc:1048] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 02:59:59.980300 8545 raft_consensus.cc:385] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.980340 8545 raft_consensus.cc:740] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.980448 8545 consensus_queue.cc:260] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.980721 8545 ts_tablet_manager.cc:1434] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 02:59:59.980931 8545 tablet_bootstrap.cc:492] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.982105 8545 tablet_bootstrap.cc:654] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.986124 7966 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 13fb883843d141f09aa90e66f9b7bf1f (table default.Table_2547f4c4cdbd4d7f9ec732da77dc545a [id=c9c59047d50a4419a9f16431bce05cd1]). Expected version 1 got 0
I20251028 02:59:59.986220 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 46ae536d269d4cbdbcee44a26d25bd67
I20251028 02:59:59.986258 7966 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 1979906599754f7ba049c7a9b23bf26d (table default.Table_cdab567cf8da46e8a4d7ba2a80e1e2d9 [id=9fcc70dd88724a28b404e9d4109015aa]). Expected version 1 got 0
I20251028 02:59:59.986488 7961 catalog_manager.cc:6719] ReplaceTablet: tablet 46ae536d269d4cbdbcee44a26d25bd67 deleted and replaced by tablet ada2c7c883014aa597ad8dbe14bf51b3
I20251028 02:59:59.987288 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 46ae536d269d4cbdbcee44a26d25bd67 with delete_type TABLET_DATA_DELETED (replaced by tablet ada2c7c883014aa597ad8dbe14bf51b3) from {username='slave'} at 127.0.0.1:46256
I20251028 02:59:59.987735 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a with delete_type TABLET_DATA_DELETED (replaced by tablet 4eaa472029a94052abb49241b7622d0f) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.987797 7965 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 464e6b3483bc4816bfa844c52fd3ca53 (table default.Table_e95d99935d7d4d94a16f5a826847ce32 [id=71026b1300014ba6a66c3431c05cc86c]). Expected version 1 got 0
W20251028 02:59:59.987852 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 46ae536d269d4cbdbcee44a26d25bd67 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46ae536d269d4cbdbcee44a26d25bd67 already in progress: creating tablet
I20251028 02:59:59.987900 8551 tablet_replica.cc:333] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 02:59:59.988047 8551 raft_consensus.cc:2243] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 02:59:59.988168 8551 raft_consensus.cc:2272] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 02:59:59.988814 8551 ts_tablet_manager.cc:1916] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 02:59:59.989023 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 46ae536d269d4cbdbcee44a26d25bd67 with delete_type TABLET_DATA_DELETED (replaced by tablet ada2c7c883014aa597ad8dbe14bf51b3) from {username='slave'} at 127.0.0.1:57400
I20251028 02:59:59.989069 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 46ae536d269d4cbdbcee44a26d25bd67 with delete_type TABLET_DATA_DELETED (replaced by tablet ada2c7c883014aa597ad8dbe14bf51b3) from {username='slave'} at 127.0.0.1:48434
I20251028 02:59:59.989681 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f13f6c5d2118406cb4485657e33bb75f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 02:59:59.989925 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b6f5cef4fc824b09ab85a48c5bd021a7" } new_table_name: "default.Table_71a85f4535144a6b98c4ff7ab835e232" modify_external_catalogs: true
W20251028 02:59:59.990106 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f13f6c5d2118406cb4485657e33bb75f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 02:59:59.991118 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6c4478bf2de14c3c824d9ab70557b237" } new_table_name: "default.Table_849a72ead23447078b8088823de4288e" modify_external_catalogs: true
I20251028 02:59:59.992719 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5277519c4a734a75bb2ec0b380676d4b
I20251028 02:59:59.992798 8551 ts_tablet_manager.cc:1929] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 02:59:59.992871 8551 log.cc:1199] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/1d5fcc9b539b46458cd8cbfc30cbcd0a
I20251028 02:59:59.992973 7968 catalog_manager.cc:6719] ReplaceTablet: tablet 5277519c4a734a75bb2ec0b380676d4b deleted and replaced by tablet b6e67d8a267b4f129131357c17387e4b
I20251028 02:59:59.993104 8551 ts_tablet_manager.cc:1950] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 02:59:59.993618 8545 tablet_bootstrap.cc:492] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 02:59:59.993705 8545 ts_tablet_manager.cc:1403] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20251028 02:59:59.993949 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]). Expected version 1 got 0
I20251028 02:59:59.994184 8545 raft_consensus.cc:359] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.994264 8545 raft_consensus.cc:385] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 02:59:59.994285 8545 raft_consensus.cc:740] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 02:59:59.994391 8545 consensus_queue.cc:260] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 02:59:59.997056 8545 ts_tablet_manager.cc:1434] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 02:59:59.997272 8545 tablet_bootstrap.cc:492] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 02:59:59.999145 8545 tablet_bootstrap.cc:654] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 02:59:59.999863 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a (table default.Table_059639c7191541caa3dbb30b333d579f [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
W20251028 03:00:00.000028 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 46ae536d269d4cbdbcee44a26d25bd67 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46ae536d269d4cbdbcee44a26d25bd67 already in progress: creating tablet
I20251028 03:00:00.000027 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 5277519c4a734a75bb2ec0b380676d4b with delete_type TABLET_DATA_DELETED (replaced by tablet b6e67d8a267b4f129131357c17387e4b) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.000319 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 5277519c4a734a75bb2ec0b380676d4b with delete_type TABLET_DATA_DELETED (replaced by tablet b6e67d8a267b4f129131357c17387e4b) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:00.000574 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 5277519c4a734a75bb2ec0b380676d4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5277519c4a734a75bb2ec0b380676d4b already in progress: creating tablet
W20251028 03:00:00.001587 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 5277519c4a734a75bb2ec0b380676d4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5277519c4a734a75bb2ec0b380676d4b already in progress: creating tablet
I20251028 03:00:00.002126 8544 tablet_bootstrap.cc:492] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.002221 8544 ts_tablet_manager.cc:1403] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20251028 03:00:00.002669 8544 raft_consensus.cc:359] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.002763 8544 raft_consensus.cc:385] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.002791 8544 raft_consensus.cc:740] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.003050 8544 consensus_queue.cc:260] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.004244 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_80603ad4c2bc4966bbc952208ce043e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.004652 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80603ad4c2bc4966bbc952208ce043e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.005836 8544 ts_tablet_manager.cc:1434] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 03:00:00.006124 8544 tablet_bootstrap.cc:492] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.008056 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 5277519c4a734a75bb2ec0b380676d4b with delete_type TABLET_DATA_DELETED (replaced by tablet b6e67d8a267b4f129131357c17387e4b) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.009096 8544 tablet_bootstrap.cc:654] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.010313 9840 raft_consensus.cc:493] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.010403 9840 raft_consensus.cc:515] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.010744 9840 leader_election.cc:290] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.011415 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6bd72bb34fe843079f8a4b828770e441" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.011787 8356 leader_election.cc:343] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.011984 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 12eecf12cac641eaad5a9b5af5890941 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.012496 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2050be1c77ea4de0a7e510783a5f4009" } modify_external_catalogs: true
I20251028 03:00:00.012665 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2050be1c77ea4de0a7e510783a5f4009" } modify_external_catalogs: true
I20251028 03:00:00.013104 8545 tablet_bootstrap.cc:492] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.013200 8545 ts_tablet_manager.cc:1403] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.001s
I20251028 03:00:00.013660 8545 raft_consensus.cc:359] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.013767 8545 raft_consensus.cc:385] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.013803 8545 raft_consensus.cc:740] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.013926 8545 consensus_queue.cc:260] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.014201 8545 ts_tablet_manager.cc:1434] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.014403 8545 tablet_bootstrap.cc:492] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.015173 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0f7ea0b3b42f4d61b7970b0dfcd52e4d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.015698 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0f7ea0b3b42f4d61b7970b0dfcd52e4d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.017565 8545 tablet_bootstrap.cc:654] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.018882 8544 tablet_bootstrap.cc:492] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.018971 8544 ts_tablet_manager.cc:1403] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.001s
I20251028 03:00:00.019456 8544 raft_consensus.cc:359] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.019557 8544 raft_consensus.cc:385] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.019588 8544 raft_consensus.cc:740] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.019711 8544 consensus_queue.cc:260] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.019992 8544 ts_tablet_manager.cc:1434] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.020625 8544 tablet_bootstrap.cc:492] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.021613 8544 tablet_bootstrap.cc:654] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.023126 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6bd72bb34fe843079f8a4b828770e441" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.023247 8144 raft_consensus.cc:2468] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:00.024360 9788 raft_consensus.cc:493] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.024447 9788 raft_consensus.cc:515] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.024782 9788 leader_election.cc:290] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.025389 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12eecf12cac641eaad5a9b5af5890941" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.025393 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12eecf12cac641eaad5a9b5af5890941" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.025458 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d077de0ccc6448cf9de68fc6f1803ce4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.025869 8083 leader_election.cc:343] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:00.025900 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d077de0ccc6448cf9de68fc6f1803ce4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:00.026015 8083 leader_election.cc:343] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.026086 8083 leader_election.cc:304] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.026218 9788 raft_consensus.cc:2749] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.030241 9840 raft_consensus.cc:493] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.030344 9840 raft_consensus.cc:515] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.030709 9840 leader_election.cc:290] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.031162 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1979906599754f7ba049c7a9b23bf26d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.031592 8545 tablet_bootstrap.cc:492] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.031700 8545 ts_tablet_manager.cc:1403] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20251028 03:00:00.031987 8544 tablet_bootstrap.cc:492] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.032106 8544 ts_tablet_manager.cc:1403] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 03:00:00.032265 8545 raft_consensus.cc:359] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.032375 8545 raft_consensus.cc:385] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.032419 8545 raft_consensus.cc:740] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.032552 8545 consensus_queue.cc:260] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.032688 8544 raft_consensus.cc:359] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.032884 8545 ts_tablet_manager.cc:1434] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.033095 8545 tablet_bootstrap.cc:492] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.033066 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 97a90c892a6c4997bae018b95f2c7e87 (DEFAULT_TABLE table=default.Table_9c2247ee8bfa4320b2c634f44cddbee9 [id=17b3da94335e41b493796b113987662f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.033317 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a90c892a6c4997bae018b95f2c7e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.034366 8396 tablet_service.cc:1505] Processing CreateTablet for tablet ab07d25dbdca4ce583e1fa9cbd5f9d47 (DEFAULT_TABLE table=default.Table_e4376f0772b24edbb1b92e585184ff59 [id=081d6aacaa8f40c2ac115ef54eba9c44]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.034623 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab07d25dbdca4ce583e1fa9cbd5f9d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.035440 8384 tablet_service.cc:1505] Processing CreateTablet for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 (DEFAULT_TABLE table=default.Table_08d693721d15425a85eb515a90b30096 [id=7e6967daf560467b8d42c95f182cb76e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.035691 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.033766 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 97a90c892a6c4997bae018b95f2c7e87 (DEFAULT_TABLE table=default.Table_9c2247ee8bfa4320b2c634f44cddbee9 [id=17b3da94335e41b493796b113987662f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.036093 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a90c892a6c4997bae018b95f2c7e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.037606 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 1c429d74609043818c89652f00a38a56 (DEFAULT_TABLE table=default.Table_a2f9a31f1859418e9dca1b761f5025e9 [id=08ff04ccebfe4cefbc475d0bb686a22f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.037839 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c429d74609043818c89652f00a38a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.038219 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 (DEFAULT_TABLE table=default.Table_08d693721d15425a85eb515a90b30096 [id=7e6967daf560467b8d42c95f182cb76e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.038419 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.040130 8251 tablet_service.cc:1505] Processing CreateTablet for tablet ab07d25dbdca4ce583e1fa9cbd5f9d47 (DEFAULT_TABLE table=default.Table_e4376f0772b24edbb1b92e585184ff59 [id=081d6aacaa8f40c2ac115ef54eba9c44]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.040213 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 97a90c892a6c4997bae018b95f2c7e87 (DEFAULT_TABLE table=default.Table_9c2247ee8bfa4320b2c634f44cddbee9 [id=17b3da94335e41b493796b113987662f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.040396 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab07d25dbdca4ce583e1fa9cbd5f9d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.035764 8127 tablet_service.cc:1505] Processing CreateTablet for tablet ab07d25dbdca4ce583e1fa9cbd5f9d47 (DEFAULT_TABLE table=default.Table_e4376f0772b24edbb1b92e585184ff59 [id=081d6aacaa8f40c2ac115ef54eba9c44]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.040582 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 (DEFAULT_TABLE table=default.Table_08d693721d15425a85eb515a90b30096 [id=7e6967daf560467b8d42c95f182cb76e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.040637 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab07d25dbdca4ce583e1fa9cbd5f9d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.040778 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.040228 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 1c429d74609043818c89652f00a38a56 (DEFAULT_TABLE table=default.Table_a2f9a31f1859418e9dca1b761f5025e9 [id=08ff04ccebfe4cefbc475d0bb686a22f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.041723 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c429d74609043818c89652f00a38a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.042821 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97a90c892a6c4997bae018b95f2c7e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.043553 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 1c429d74609043818c89652f00a38a56 (DEFAULT_TABLE table=default.Table_a2f9a31f1859418e9dca1b761f5025e9 [id=08ff04ccebfe4cefbc475d0bb686a22f]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 03:00:00.043725 8356 leader_election.cc:343] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.043845 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c429d74609043818c89652f00a38a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.046134 8545 tablet_bootstrap.cc:654] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.032819 8544 raft_consensus.cc:385] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.046490 8544 raft_consensus.cc:740] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.046689 8544 consensus_queue.cc:260] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.047088 8544 ts_tablet_manager.cc:1434] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.015s user 0.001s sys 0.000s
I20251028 03:00:00.047315 8544 tablet_bootstrap.cc:492] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.048903 8544 tablet_bootstrap.cc:654] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.048957 8546 tablet_bootstrap.cc:492] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.049058 8546 ts_tablet_manager.cc:1403] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.135s user 0.002s sys 0.001s
I20251028 03:00:00.049427 9788 raft_consensus.cc:493] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.049506 9788 raft_consensus.cc:515] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.049506 8546 raft_consensus.cc:359] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.049633 8546 raft_consensus.cc:385] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.049662 8546 raft_consensus.cc:740] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.049813 8546 consensus_queue.cc:260] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.049892 9788 leader_election.cc:290] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.050077 8546 ts_tablet_manager.cc:1434] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.050267 8546 tablet_bootstrap.cc:492] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.051113 8546 tablet_bootstrap.cc:654] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.053665 8544 tablet_bootstrap.cc:492] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.053748 8544 ts_tablet_manager.cc:1403] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 03:00:00.054203 8544 raft_consensus.cc:359] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.054289 8544 raft_consensus.cc:385] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.054318 8544 raft_consensus.cc:740] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.054440 8544 consensus_queue.cc:260] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.054693 8544 ts_tablet_manager.cc:1434] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.054852 8544 tablet_bootstrap.cc:492] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.055871 8544 tablet_bootstrap.cc:654] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.058035 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b9b5fb938357499b8888ba29d23612bd" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.058508 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b9b5fb938357499b8888ba29d23612bd" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.058851 8083 leader_election.cc:343] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:00.059033 8083 leader_election.cc:343] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.059089 8083 leader_election.cc:304] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.059554 9788 raft_consensus.cc:2749] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.061247 8546 tablet_bootstrap.cc:492] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.061345 8546 ts_tablet_manager.cc:1403] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20251028 03:00:00.061769 8546 raft_consensus.cc:359] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.061877 8546 raft_consensus.cc:385] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.061913 8546 raft_consensus.cc:740] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.062031 8546 consensus_queue.cc:260] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.062534 8546 ts_tablet_manager.cc:1434] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.062875 8546 tablet_bootstrap.cc:492] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.063973 8544 tablet_bootstrap.cc:492] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.064077 8544 ts_tablet_manager.cc:1403] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20251028 03:00:00.064491 8544 raft_consensus.cc:359] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.064594 8544 raft_consensus.cc:385] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.064631 8544 raft_consensus.cc:740] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.064632 8546 tablet_bootstrap.cc:654] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.064760 8544 consensus_queue.cc:260] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.065025 8544 ts_tablet_manager.cc:1434] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.065208 8544 tablet_bootstrap.cc:492] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.066140 8544 tablet_bootstrap.cc:654] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.066232 8546 tablet_bootstrap.cc:492] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.066311 8546 ts_tablet_manager.cc:1403] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20251028 03:00:00.067632 8546 raft_consensus.cc:359] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.069224 8546 raft_consensus.cc:385] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.069298 8546 raft_consensus.cc:740] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.069455 8546 consensus_queue.cc:260] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.076474 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1979906599754f7ba049c7a9b23bf26d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.076534 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5a3394a1e1254df19e6e3f6641de825a
I20251028 03:00:00.076622 8144 raft_consensus.cc:2468] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:00.077737 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 1d3e422c7ae64b25960714ecc926046a (table default.Table_d9b8ba18e24848b1b8511a66b8c7cf2f [id=181b50c2d29a4b17b8189e3abb10a420]). Expected version 2 got 0
I20251028 03:00:00.079967 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 5a3394a1e1254df19e6e3f6641de825a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.080035 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 81c2c40e4da242f8891687a6365d273c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.080318 8551 tablet_replica.cc:333] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.080471 8551 raft_consensus.cc:2243] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.080592 8551 raft_consensus.cc:2272] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:00.080969 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 5a3394a1e1254df19e6e3f6641de825a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a3394a1e1254df19e6e3f6641de825a already in progress: creating tablet
I20251028 03:00:00.081142 8551 ts_tablet_manager.cc:1916] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.083114 8550 ts_tablet_manager.cc:1929] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:00.083560 8551 ts_tablet_manager.cc:1929] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.083644 8551 log.cc:1199] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/81c2c40e4da242f8891687a6365d273c
I20251028 03:00:00.084816 8551 ts_tablet_manager.cc:1950] T 81c2c40e4da242f8891687a6365d273c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.085023 8354 leader_election.cc:304] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:00.085237 8354 leader_election.cc:304] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:00.085510 9190 raft_consensus.cc:2804] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.085544 9639 raft_consensus.cc:2804] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.085570 9190 raft_consensus.cc:493] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.085551 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 81c2c40e4da242f8891687a6365d273c (table default.Table_5eddcfdf60ef439397a4948da2d85f90 [id=1cf0d713d3de40858825646f084e8505]) successfully deleted
I20251028 03:00:00.085598 9639 raft_consensus.cc:493] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.085613 9190 raft_consensus.cc:3060] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.085639 9639 raft_consensus.cc:3060] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.086545 9639 raft_consensus.cc:515] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.086633 9190 raft_consensus.cc:515] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.086902 9639 leader_election.cc:290] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.086938 9190 leader_election.cc:290] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.087538 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1979906599754f7ba049c7a9b23bf26d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:00.087939 8356 leader_election.cc:343] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.088132 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6bd72bb34fe843079f8a4b828770e441" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:00.088485 8356 leader_election.cc:343] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.091518 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 5a3394a1e1254df19e6e3f6641de825a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:00.091971 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 5a3394a1e1254df19e6e3f6641de825a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a3394a1e1254df19e6e3f6641de825a already in progress: creating tablet
I20251028 03:00:00.084972 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 95f4e2921c97438ab9db2878d85cf66e (DEFAULT_TABLE table=default.Table_c158a2a1e7d14570a486cdbbc6044efd [id=40f8798a68d54ea991ef9bf6887dfe36]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.093420 8550 log.cc:1199] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/0e20b1f36d3f4f70a93ad8a0b29d6687
I20251028 03:00:00.093604 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95f4e2921c97438ab9db2878d85cf66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.094386 8550 ts_tablet_manager.cc:1950] T 0e20b1f36d3f4f70a93ad8a0b29d6687 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.094995 8550 tablet_replica.cc:333] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.095165 8550 raft_consensus.cc:2243] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.095257 8550 raft_consensus.cc:2272] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.095432 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5fa32a1433cd47a9ad39d1c613a18761
I20251028 03:00:00.095706 8544 tablet_bootstrap.cc:492] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.095785 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 5fa32a1433cd47a9ad39d1c613a18761 deleted and replaced by tablet dfda4b9d46ae427baede4b8b44c2a843
I20251028 03:00:00.095806 8544 ts_tablet_manager.cc:1403] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.031s user 0.000s sys 0.001s
I20251028 03:00:00.096030 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fa32a1433cd47a9ad39d1c613a18761 with delete_type TABLET_DATA_DELETED (replaced by tablet dfda4b9d46ae427baede4b8b44c2a843) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.096287 8544 raft_consensus.cc:359] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.096422 8544 raft_consensus.cc:385] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.096464 8544 raft_consensus.cc:740] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.096586 8544 consensus_queue.cc:260] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
W20251028 03:00:00.096752 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 5fa32a1433cd47a9ad39d1c613a18761 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fa32a1433cd47a9ad39d1c613a18761 already in progress: creating tablet
I20251028 03:00:00.096890 8544 ts_tablet_manager.cc:1434] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.097054 8544 tablet_bootstrap.cc:492] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.097193 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet c120a3d56f6b41c28c7f7dcd84359a60 (table default.Table_13dbc4fadcfd41c8945172798836fb62 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]). Expected version 1 got 0
I20251028 03:00:00.097286 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 1d3e422c7ae64b25960714ecc926046a (table default.Table_d9b8ba18e24848b1b8511a66b8c7cf2f [id=181b50c2d29a4b17b8189e3abb10a420]). Expected version 2 got 0
I20251028 03:00:00.097623 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 3727b09625f948d5acb735d4e88fe948 (DEFAULT_TABLE table=default.Table_a19c30e391a34b00816d39f3426a855d [id=6e144d0a3fa244aea334391ed6ed5917]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.097793 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 680f8235c6e64a48bd0009c699760691
I20251028 03:00:00.097915 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3727b09625f948d5acb735d4e88fe948. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.098280 8544 tablet_bootstrap.cc:654] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.098876 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_594adc002945439ba87c71677fcb4ad1" } new_table_name: "default.Table_5ba8c01948c043be8f3f6f6112a40078" modify_external_catalogs: true
I20251028 03:00:00.099924 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_316b63d9aec64a06b194dca118966b47" } new_table_name: "default.Table_4bafc1e32c454d84b181dadf3727efcc" modify_external_catalogs: true
I20251028 03:00:00.101104 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 680f8235c6e64a48bd0009c699760691 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.101253 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 0e20b1f36d3f4f70a93ad8a0b29d6687 (table default.Table_7778bd93ae56485f921a20d0df78a3ec [id=7a2a6ab3c9744942a9413c6cb057f527]) successfully deleted
W20251028 03:00:00.101567 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 680f8235c6e64a48bd0009c699760691 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 680f8235c6e64a48bd0009c699760691 already in progress: creating tablet
I20251028 03:00:00.101699 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fa32a1433cd47a9ad39d1c613a18761 with delete_type TABLET_DATA_DELETED (replaced by tablet dfda4b9d46ae427baede4b8b44c2a843) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.101743 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 680f8235c6e64a48bd0009c699760691 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.102200 9595 raft_consensus.cc:493] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.102303 9595 raft_consensus.cc:515] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.102746 9595 leader_election.cc:290] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.103236 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fa32a1433cd47a9ad39d1c613a18761 with delete_type TABLET_DATA_DELETED (replaced by tablet dfda4b9d46ae427baede4b8b44c2a843) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.103350 8384 tablet_service.cc:1505] Processing CreateTablet for tablet 95f4e2921c97438ab9db2878d85cf66e (DEFAULT_TABLE table=default.Table_c158a2a1e7d14570a486cdbbc6044efd [id=40f8798a68d54ea991ef9bf6887dfe36]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.103408 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 3727b09625f948d5acb735d4e88fe948 (DEFAULT_TABLE table=default.Table_a19c30e391a34b00816d39f3426a855d [id=6e144d0a3fa244aea334391ed6ed5917]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.103540 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet 680f8235c6e64a48bd0009c699760691 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.103657 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95f4e2921c97438ab9db2878d85cf66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.103818 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3dae09d02c134e68bf3159a5dc3d3053"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.104149 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3dae09d02c134e68bf3159a5dc3d3053 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.104377 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 97c0ebd67d0141ee9c823602213d0b2e (DEFAULT_TABLE table=default.Table_94da0c3c823c4016b6534ddecabf64a9 [id=1a5265ee014b4e4d9ac9bb7e820857a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.104607 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97c0ebd67d0141ee9c823602213d0b2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.105685 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 3727b09625f948d5acb735d4e88fe948 (DEFAULT_TABLE table=default.Table_a19c30e391a34b00816d39f3426a855d [id=6e144d0a3fa244aea334391ed6ed5917]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.105939 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3727b09625f948d5acb735d4e88fe948. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.106861 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3727b09625f948d5acb735d4e88fe948. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.107445 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 97c0ebd67d0141ee9c823602213d0b2e (DEFAULT_TABLE table=default.Table_94da0c3c823c4016b6534ddecabf64a9 [id=1a5265ee014b4e4d9ac9bb7e820857a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.107661 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97c0ebd67d0141ee9c823602213d0b2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.107733 8550 ts_tablet_manager.cc:1916] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.108556 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 97c0ebd67d0141ee9c823602213d0b2e (DEFAULT_TABLE table=default.Table_94da0c3c823c4016b6534ddecabf64a9 [id=1a5265ee014b4e4d9ac9bb7e820857a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.108800 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97c0ebd67d0141ee9c823602213d0b2e. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:00.109633 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 5fa32a1433cd47a9ad39d1c613a18761 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fa32a1433cd47a9ad39d1c613a18761 already in progress: creating tablet
W20251028 03:00:00.109848 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 680f8235c6e64a48bd0009c699760691 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 680f8235c6e64a48bd0009c699760691 already in progress: creating tablet
I20251028 03:00:00.111902 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13fb883843d141f09aa90e66f9b7bf1f" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.112032 8416 raft_consensus.cc:2468] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:00.112046 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 95f4e2921c97438ab9db2878d85cf66e (DEFAULT_TABLE table=default.Table_c158a2a1e7d14570a486cdbbc6044efd [id=40f8798a68d54ea991ef9bf6887dfe36]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.112315 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95f4e2921c97438ab9db2878d85cf66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.113742 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 9d438110cfec42a991b6c40b0881cc62 (DEFAULT_TABLE table=default.Table_4fe6d656356041dcaad42a9fd6f026d5 [id=f8375f32144c4541bea5dbcd51ab256e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.114012 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d438110cfec42a991b6c40b0881cc62. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.114897 8220 leader_election.cc:304] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.091744 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 5a3394a1e1254df19e6e3f6641de825a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.115458 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 9d438110cfec42a991b6c40b0881cc62 (DEFAULT_TABLE table=default.Table_4fe6d656356041dcaad42a9fd6f026d5 [id=f8375f32144c4541bea5dbcd51ab256e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.115692 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d438110cfec42a991b6c40b0881cc62. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.116232 7968 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]). Expected version 1 got 0
I20251028 03:00:00.116699 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7f8968a27516426095ca46f368ba84b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.117058 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f8968a27516426095ca46f368ba84b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:00.118046 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7ee583e318174f34b162c2a104a0a3a9 consensus state has no leader
I20251028 03:00:00.118252 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_df4ca62b1a17423d8e824e6eb889a2c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.118547 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df4ca62b1a17423d8e824e6eb889a2c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.119179 9958 raft_consensus.cc:2804] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.119254 9958 raft_consensus.cc:493] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.119336 9958 raft_consensus.cc:3060] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.119987 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c175a83383134da183a6ca9e861ea2ba" } new_table_name: "default.Table_e9c0af79b3c44383883fa3c5ecfa256d" modify_external_catalogs: true
I20251028 03:00:00.122871 8128 tablet_service.cc:1505] Processing CreateTablet for tablet d5b660efe0bc4feea0c09383ad2aa682 (DEFAULT_TABLE table=default.Table_acbe587ace4042f29f736a17a6d2530a [id=c5de3c9bf3524daf845f65158d3b14b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.123394 8256 tablet_service.cc:1505] Processing CreateTablet for tablet d5b660efe0bc4feea0c09383ad2aa682 (DEFAULT_TABLE table=default.Table_acbe587ace4042f29f736a17a6d2530a [id=c5de3c9bf3524daf845f65158d3b14b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.123667 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b660efe0bc4feea0c09383ad2aa682. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:00.124415 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2b6f62b942a94af783c026a043cc60a3 consensus state has no leader
I20251028 03:00:00.124625 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0ce46dd2a59848399a1d4fdeac7deef9" } modify_external_catalogs: true
I20251028 03:00:00.124722 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0ce46dd2a59848399a1d4fdeac7deef9" } modify_external_catalogs: true
I20251028 03:00:00.128763 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_9f07958f85cf4923a67bf8ac1ad1c722" } modify_external_catalogs: true
I20251028 03:00:00.128890 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_9f07958f85cf4923a67bf8ac1ad1c722" } modify_external_catalogs: true
I20251028 03:00:00.129608 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet b42f039c30a048e299c39bcf44033125
I20251028 03:00:00.131486 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 9d438110cfec42a991b6c40b0881cc62 (DEFAULT_TABLE table=default.Table_4fe6d656356041dcaad42a9fd6f026d5 [id=f8375f32144c4541bea5dbcd51ab256e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.131790 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d438110cfec42a991b6c40b0881cc62. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.132184 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet ffe824a618714776bfab38679c9565e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.132539 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet b42f039c30a048e299c39bcf44033125 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:00.133258 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet b42f039c30a048e299c39bcf44033125 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b42f039c30a048e299c39bcf44033125 already in progress: creating tablet
I20251028 03:00:00.133792 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet b42f039c30a048e299c39bcf44033125 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.134670 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet b9803f4cfb9b4ef9a1954c83985d6258 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b6f62b942a94af783c026a043cc60a3) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.136132 8398 tablet_service.cc:1552] Processing DeleteTablet for tablet b42f039c30a048e299c39bcf44033125 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:00.136612 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet b42f039c30a048e299c39bcf44033125 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b42f039c30a048e299c39bcf44033125 already in progress: creating tablet
I20251028 03:00:00.139243 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 with delete_type TABLET_DATA_DELETED (replaced by tablet 751777e4aac9489183e021c2367d3122) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:00.139650 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a5415a4246c47ca896751f2fd68af71 consensus state has no leader
W20251028 03:00:00.141515 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fbf0114b95a4492380d4048d8f4c97e4 consensus state has no leader
I20251028 03:00:00.141914 8546 ts_tablet_manager.cc:1434] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.076s user 0.001s sys 0.000s
I20251028 03:00:00.142145 8546 tablet_bootstrap.cc:492] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.144166 9958 raft_consensus.cc:515] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.145378 8398 tablet_service.cc:1505] Processing CreateTablet for tablet d5b660efe0bc4feea0c09383ad2aa682 (DEFAULT_TABLE table=default.Table_acbe587ace4042f29f736a17a6d2530a [id=c5de3c9bf3524daf845f65158d3b14b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.145601 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b660efe0bc4feea0c09383ad2aa682. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.145782 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b660efe0bc4feea0c09383ad2aa682. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.147573 9913 tablet.cc:1722] T 09d65eef81b2421c92678015b6da4fbb P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:00.149143 8550 ts_tablet_manager.cc:1929] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.150424 8544 tablet_bootstrap.cc:492] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.150424 8545 tablet_bootstrap.cc:492] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.150518 8545 ts_tablet_manager.cc:1403] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.117s user 0.000s sys 0.002s
I20251028 03:00:00.150518 8544 ts_tablet_manager.cc:1403] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.054s user 0.001s sys 0.001s
I20251028 03:00:00.150981 8545 raft_consensus.cc:359] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.150981 8544 raft_consensus.cc:359] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.151088 8545 raft_consensus.cc:385] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.151088 8544 raft_consensus.cc:385] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.151124 8545 raft_consensus.cc:740] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.151124 8544 raft_consensus.cc:740] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.151244 8544 consensus_queue.cc:260] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.151244 8545 consensus_queue.cc:260] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.151558 8545 ts_tablet_manager.cc:1434] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.151566 8544 ts_tablet_manager.cc:1434] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.151752 8545 tablet_bootstrap.cc:492] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.151762 8544 tablet_bootstrap.cc:492] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.151741 8546 tablet_bootstrap.cc:654] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.152693 8545 tablet_bootstrap.cc:654] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.152694 8544 tablet_bootstrap.cc:654] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.153795 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1979906599754f7ba049c7a9b23bf26d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.153905 8144 raft_consensus.cc:3060] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.154415 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 13fb883843d141f09aa90e66f9b7bf1f (table default.Table_2547f4c4cdbd4d7f9ec732da77dc545a [id=c9c59047d50a4419a9f16431bce05cd1]). Expected version 1 got 0
I20251028 03:00:00.154558 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]). Expected version 1 got 0
I20251028 03:00:00.154846 8144 raft_consensus.cc:2468] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.155099 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6bd72bb34fe843079f8a4b828770e441" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.155196 8146 raft_consensus.cc:3060] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.155620 9958 leader_election.cc:290] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.156127 8146 raft_consensus.cc:2468] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.156121 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13fb883843d141f09aa90e66f9b7bf1f" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:00.156231 8419 raft_consensus.cc:3060] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.157209 8419 raft_consensus.cc:2468] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:00.157567 8220 leader_election.cc:304] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.157907 8550 log.cc:1199] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/0e19a2bdc7024b0c9fe84f8983f9d292
I20251028 03:00:00.157795 9958 raft_consensus.cc:2804] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.158196 8550 ts_tablet_manager.cc:1950] T 0e19a2bdc7024b0c9fe84f8983f9d292 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.158878 9958 raft_consensus.cc:697] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:00.159015 8550 tablet_replica.cc:333] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.159065 9958 consensus_queue.cc:237] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.159828 9906 tablet.cc:1722] T 09d65eef81b2421c92678015b6da4fbb P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:00.160566 8550 raft_consensus.cc:2243] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.160676 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13fb883843d141f09aa90e66f9b7bf1f" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.160826 8145 raft_consensus.cc:2468] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:00.160674 8550 raft_consensus.cc:2272] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.161437 8550 ts_tablet_manager.cc:1916] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.162247 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13fb883843d141f09aa90e66f9b7bf1f" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.162343 8143 raft_consensus.cc:3060] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.163336 8143 raft_consensus.cc:2468] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:00.163870 8550 ts_tablet_manager.cc:1929] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.163969 8550 log.cc:1199] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/12eecf12cac641eaad5a9b5af5890941
I20251028 03:00:00.164208 8550 ts_tablet_manager.cc:1950] T 12eecf12cac641eaad5a9b5af5890941 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.164381 9958 consensus_queue.cc:1048] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.165091 8550 tablet_replica.cc:333] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.165293 8550 raft_consensus.cc:2243] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.165477 8550 raft_consensus.cc:2272] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.166002 8550 ts_tablet_manager.cc:1916] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.167901 9909 tablet.cc:1722] T 09d65eef81b2421c92678015b6da4fbb P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:00.168124 8546 tablet_bootstrap.cc:492] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.168186 8550 ts_tablet_manager.cc:1929] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.168215 8546 ts_tablet_manager.cc:1403] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.002s
I20251028 03:00:00.168282 8550 log.cc:1199] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ffe824a618714776bfab38679c9565e0
I20251028 03:00:00.168534 8550 ts_tablet_manager.cc:1950] T ffe824a618714776bfab38679c9565e0 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.168771 8546 raft_consensus.cc:359] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.168898 8546 raft_consensus.cc:385] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.168938 8546 raft_consensus.cc:740] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.169070 8546 consensus_queue.cc:260] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.169230 8550 tablet_replica.cc:333] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.169359 8550 raft_consensus.cc:2243] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.169385 8546 ts_tablet_manager.cc:1434] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.169445 8550 raft_consensus.cc:2272] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.169574 8546 tablet_bootstrap.cc:492] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.169961 8550 ts_tablet_manager.cc:1916] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.170682 8546 tablet_bootstrap.cc:654] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.170789 8273 raft_consensus.cc:3060] T 94cbad596db44060bac2992e9ed17ed8 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.173851 8550 ts_tablet_manager.cc:1929] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.174026 8550 log.cc:1199] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/b9803f4cfb9b4ef9a1954c83985d6258
I20251028 03:00:00.174312 8550 ts_tablet_manager.cc:1950] T b9803f4cfb9b4ef9a1954c83985d6258 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.174968 8550 tablet_replica.cc:333] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.175165 8550 raft_consensus.cc:2243] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.175346 8550 raft_consensus.cc:2272] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.176029 8546 tablet_bootstrap.cc:492] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.176131 8546 ts_tablet_manager.cc:1403] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20251028 03:00:00.176697 8546 raft_consensus.cc:359] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.176811 8546 raft_consensus.cc:385] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.176842 8546 raft_consensus.cc:740] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.176985 8546 consensus_queue.cc:260] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.177304 8546 ts_tablet_manager.cc:1434] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.177505 8546 tablet_bootstrap.cc:492] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.178850 9913 tablet.cc:1722] T 94cbad596db44060bac2992e9ed17ed8 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:00.179919 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_71f1dfc334c447899b2d951d85d4547a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.180071 8546 tablet_bootstrap.cc:654] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:00.180405 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_71f1dfc334c447899b2d951d85d4547a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.181401 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_86557fb2d1974f27bee360a4c8355497"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.181699 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86557fb2d1974f27bee360a4c8355497 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.181790 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 0e19a2bdc7024b0c9fe84f8983f9d292 (table default.Table_4f7a6b08bb2b4cee90e54afd8f1b801a [id=243b57c791df41819156363e8607623d]) successfully deleted
W20251028 03:00:00.181959 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 46ae536d269d4cbdbcee44a26d25bd67 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46ae536d269d4cbdbcee44a26d25bd67 already in progress: creating tablet
W20251028 03:00:00.182077 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 5277519c4a734a75bb2ec0b380676d4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5277519c4a734a75bb2ec0b380676d4b already in progress: creating tablet
I20251028 03:00:00.182135 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 12eecf12cac641eaad5a9b5af5890941 (table default.Table_7ceb18d43d6c41d2b7809c8c97a5f2e7 [id=e47909e4e17049549b87f01157c35a4f]) successfully deleted
W20251028 03:00:00.182196 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 5fa32a1433cd47a9ad39d1c613a18761 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fa32a1433cd47a9ad39d1c613a18761 already in progress: creating tablet
W20251028 03:00:00.182256 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 680f8235c6e64a48bd0009c699760691 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 680f8235c6e64a48bd0009c699760691 already in progress: creating tablet
W20251028 03:00:00.182312 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 5a3394a1e1254df19e6e3f6641de825a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a3394a1e1254df19e6e3f6641de825a already in progress: creating tablet
I20251028 03:00:00.182322 8546 tablet_bootstrap.cc:492] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.182384 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet ffe824a618714776bfab38679c9565e0 (table default.Table_6f0cc2934b86479083615ebdb7ead649 [id=652e99fe1390470387f6f57273e4e48b]) successfully deleted
I20251028 03:00:00.182405 8546 ts_tablet_manager.cc:1403] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20251028 03:00:00.182411 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet 12d103e9b5024a87aa232c26e0c51ab9 from {username='slave'} at 127.0.0.1:47364
W20251028 03:00:00.182475 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet b42f039c30a048e299c39bcf44033125 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b42f039c30a048e299c39bcf44033125 already in progress: creating tablet
I20251028 03:00:00.182578 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet b9803f4cfb9b4ef9a1954c83985d6258 (table default.Table_849a72ead23447078b8088823de4288e [id=82ae3d70b0b544d89bd8e0ff1717f396]) successfully deleted
I20251028 03:00:00.182734 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet c120a3d56f6b41c28c7f7dcd84359a60 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:00.182929 8546 raft_consensus.cc:359] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.183048 8546 raft_consensus.cc:385] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.183080 8546 raft_consensus.cc:740] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.183187 8546 consensus_queue.cc:260] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.183228 8354 leader_election.cc:304] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:00.183517 8354 leader_election.cc:304] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:00.183558 8546 ts_tablet_manager.cc:1434] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251028 03:00:00.185097 9190 raft_consensus.cc:2804] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.185192 9190 raft_consensus.cc:697] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:00.185330 9190 consensus_queue.cc:237] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.185400 8546 tablet_bootstrap.cc:492] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.185933 9190 raft_consensus.cc:2804] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.186022 9190 raft_consensus.cc:697] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:00.186132 9190 consensus_queue.cc:237] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.187036 8546 tablet_bootstrap.cc:654] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.188198 8546 tablet_bootstrap.cc:492] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.188315 8546 ts_tablet_manager.cc:1403] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251028 03:00:00.188840 8546 raft_consensus.cc:359] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.188980 8546 raft_consensus.cc:385] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.189033 8546 raft_consensus.cc:740] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.189144 8546 consensus_queue.cc:260] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.189435 8546 ts_tablet_manager.cc:1434] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.189743 9958 raft_consensus.cc:493] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.189833 9958 raft_consensus.cc:515] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.190161 9958 leader_election.cc:290] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:00.190194 8546 tablet_bootstrap.cc:492] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 03:00:00.190878 8218 leader_election.cc:343] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 1d5fcc9b539b46458cd8cbfc30cbcd0a
I20251028 03:00:00.191215 8546 tablet_bootstrap.cc:654] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:00.193310 8220 leader_election.cc:343] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 1d5fcc9b539b46458cd8cbfc30cbcd0a
I20251028 03:00:00.193388 8220 leader_election.cc:304] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:00.193472 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4393587f022446828e344a9c394a92f9
I20251028 03:00:00.193540 9958 raft_consensus.cc:2749] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.193905 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2ff112c66f1a4cf9b232e6bb665755f9
I20251028 03:00:00.194051 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 4393587f022446828e344a9c394a92f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.194319 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ff112c66f1a4cf9b232e6bb665755f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.194669 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ff112c66f1a4cf9b232e6bb665755f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:00.194566 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4393587f022446828e344a9c394a92f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4393587f022446828e344a9c394a92f9 already in progress: creating tablet
W20251028 03:00:00.194904 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 2ff112c66f1a4cf9b232e6bb665755f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ff112c66f1a4cf9b232e6bb665755f9 already in progress: creating tablet
W20251028 03:00:00.195073 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 2ff112c66f1a4cf9b232e6bb665755f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ff112c66f1a4cf9b232e6bb665755f9 already in progress: creating tablet
I20251028 03:00:00.196507 8546 tablet_bootstrap.cc:492] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.196612 8546 ts_tablet_manager.cc:1403] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20251028 03:00:00.196724 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 4393587f022446828e344a9c394a92f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.197223 8546 raft_consensus.cc:359] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.197319 8546 raft_consensus.cc:385] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.197350 8546 raft_consensus.cc:740] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.197463 8546 consensus_queue.cc:260] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.197690 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 4393587f022446828e344a9c394a92f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.197764 8546 ts_tablet_manager.cc:1434] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.197798 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ff112c66f1a4cf9b232e6bb665755f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:00.199457 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4393587f022446828e344a9c394a92f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4393587f022446828e344a9c394a92f9 already in progress: creating tablet
I20251028 03:00:00.199712 8546 tablet_bootstrap.cc:492] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.200862 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c120a3d56f6b41c28c7f7dcd84359a60
I20251028 03:00:00.201102 7967 catalog_manager.cc:6719] ReplaceTablet: tablet c120a3d56f6b41c28c7f7dcd84359a60 deleted and replaced by tablet dea4de43bc384ac8bd55472aa1e59c7e
I20251028 03:00:00.201184 8546 tablet_bootstrap.cc:654] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.201856 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet c120a3d56f6b41c28c7f7dcd84359a60 with delete_type TABLET_DATA_DELETED (replaced by tablet dea4de43bc384ac8bd55472aa1e59c7e) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.202330 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet c120a3d56f6b41c28c7f7dcd84359a60 with delete_type TABLET_DATA_DELETED (replaced by tablet dea4de43bc384ac8bd55472aa1e59c7e) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.203349 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_9032b689a6154ff8bfd6bb7c5cfc1373"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.203905 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9032b689a6154ff8bfd6bb7c5cfc1373 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.207173 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f04c867f0ce4ab99f980cca063abc30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.207777 8550 ts_tablet_manager.cc:1916] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.208087 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet c120a3d56f6b41c28c7f7dcd84359a60 with delete_type TABLET_DATA_DELETED (replaced by tablet dea4de43bc384ac8bd55472aa1e59c7e) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.208401 8552 tablet_replica.cc:333] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.208530 8552 raft_consensus.cc:2243] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.208634 8552 raft_consensus.cc:2272] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:00.208881 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet c120a3d56f6b41c28c7f7dcd84359a60 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c120a3d56f6b41c28c7f7dcd84359a60 already in progress: creating tablet
W20251028 03:00:00.209049 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet c120a3d56f6b41c28c7f7dcd84359a60 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c120a3d56f6b41c28c7f7dcd84359a60 already in progress: creating tablet
I20251028 03:00:00.209358 8552 ts_tablet_manager.cc:1916] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.210294 8550 ts_tablet_manager.cc:1929] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.210368 8550 log.cc:1199] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/fa70b61f9c8840b8b8f1b0f0a2ec0a17
I20251028 03:00:00.210597 8550 ts_tablet_manager.cc:1950] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.211153 8550 tablet_replica.cc:333] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.211733 8552 ts_tablet_manager.cc:1929] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.211869 8552 log.cc:1199] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3f04c867f0ce4ab99f980cca063abc30
I20251028 03:00:00.211282 8550 raft_consensus.cc:2243] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.211961 8546 tablet_bootstrap.cc:492] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.212040 8550 raft_consensus.cc:2272] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.212070 8546 ts_tablet_manager.cc:1403] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20251028 03:00:00.212414 8552 ts_tablet_manager.cc:1950] T 3f04c867f0ce4ab99f980cca063abc30 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.212613 8546 raft_consensus.cc:359] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.212744 8546 raft_consensus.cc:385] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.212788 8546 raft_consensus.cc:740] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.212949 8546 consensus_queue.cc:260] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.213261 8546 ts_tablet_manager.cc:1434] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.213436 8546 tablet_bootstrap.cc:492] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.214462 8546 tablet_bootstrap.cc:654] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.215029 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3f04c867f0ce4ab99f980cca063abc30 (table default.Table_7ac21b2838614cc3bce8e42fe5e3e5d1 [id=3940830838ac4130bcecb88884e4ea64]) successfully deleted
I20251028 03:00:00.215354 8550 ts_tablet_manager.cc:1916] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.215656 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 (table default.Table_71dd113f5d12423d9aca617205523aa1 [id=9cd25ed7b9a04064b29a42f6fcd95e00]) successfully deleted
W20251028 03:00:00.217196 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4393587f022446828e344a9c394a92f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4393587f022446828e344a9c394a92f9 already in progress: creating tablet
I20251028 03:00:00.217757 8544 tablet_bootstrap.cc:492] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.217861 8544 ts_tablet_manager.cc:1403] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.066s user 0.001s sys 0.000s
W20251028 03:00:00.218226 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 2ff112c66f1a4cf9b232e6bb665755f9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ff112c66f1a4cf9b232e6bb665755f9 already in progress: creating tablet
I20251028 03:00:00.218353 8544 raft_consensus.cc:359] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.218464 8544 raft_consensus.cc:385] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.218503 8544 raft_consensus.cc:740] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.218649 8544 consensus_queue.cc:260] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.218935 8544 ts_tablet_manager.cc:1434] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.219139 8544 tablet_bootstrap.cc:492] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.220201 8544 tablet_bootstrap.cc:654] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.223933 7962 catalog_manager.cc:5649] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:00.224205 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 13fb883843d141f09aa90e66f9b7bf1f (table default.Table_2547f4c4cdbd4d7f9ec732da77dc545a [id=c9c59047d50a4419a9f16431bce05cd1]). Expected version 1 got 0
I20251028 03:00:00.225894 8384 tablet_service.cc:1505] Processing CreateTablet for tablet 87c0f1accc254ee19372bcff9a71ed5d (DEFAULT_TABLE table=default.Table_0f7ea0b3b42f4d61b7970b0dfcd52e4d [id=acf2b2ad89e043e799d7540f42ddd192]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.225896 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 87c0f1accc254ee19372bcff9a71ed5d (DEFAULT_TABLE table=default.Table_0f7ea0b3b42f4d61b7970b0dfcd52e4d [id=acf2b2ad89e043e799d7540f42ddd192]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.226197 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87c0f1accc254ee19372bcff9a71ed5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.226197 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87c0f1accc254ee19372bcff9a71ed5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.228135 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 87c0f1accc254ee19372bcff9a71ed5d (DEFAULT_TABLE table=default.Table_0f7ea0b3b42f4d61b7970b0dfcd52e4d [id=acf2b2ad89e043e799d7540f42ddd192]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.228931 8128 tablet_service.cc:1505] Processing CreateTablet for tablet e9b5228d4131446f8460917e961b8f15 (DEFAULT_TABLE table=default.Table_3dae09d02c134e68bf3159a5dc3d3053 [id=eed7ed4ee0e4411592a96ca563223506]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.229084 8261 tablet_service.cc:1505] Processing CreateTablet for tablet e9b5228d4131446f8460917e961b8f15 (DEFAULT_TABLE table=default.Table_3dae09d02c134e68bf3159a5dc3d3053 [id=eed7ed4ee0e4411592a96ca563223506]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.229355 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b5228d4131446f8460917e961b8f15. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.229861 8384 tablet_service.cc:1505] Processing CreateTablet for tablet e9b5228d4131446f8460917e961b8f15 (DEFAULT_TABLE table=default.Table_3dae09d02c134e68bf3159a5dc3d3053 [id=eed7ed4ee0e4411592a96ca563223506]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.230104 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b5228d4131446f8460917e961b8f15. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.230392 8545 tablet_bootstrap.cc:492] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.230469 8545 ts_tablet_manager.cc:1403] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.079s user 0.000s sys 0.001s
I20251028 03:00:00.230917 8545 raft_consensus.cc:359] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.231048 8545 raft_consensus.cc:385] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.231083 8545 raft_consensus.cc:740] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.231211 8545 consensus_queue.cc:260] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.231551 8545 ts_tablet_manager.cc:1434] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.231745 8545 tablet_bootstrap.cc:492] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.232367 8550 ts_tablet_manager.cc:1929] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.232481 8550 log.cc:1199] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c120a3d56f6b41c28c7f7dcd84359a60
I20251028 03:00:00.232741 8545 tablet_bootstrap.cc:654] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.232980 8550 ts_tablet_manager.cc:1950] T c120a3d56f6b41c28c7f7dcd84359a60 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.234407 7960 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]). Expected version 1 got 0
I20251028 03:00:00.234606 8111 tablet_service.cc:1505] Processing CreateTablet for tablet fb59e5f7bb57488ba2f9e4b43a118bf8 (DEFAULT_TABLE table=default.Table_80603ad4c2bc4966bbc952208ce043e2 [id=a222e22ca83a46f793259accf7b4bfbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.235960 8127 tablet_service.cc:1505] Processing CreateTablet for tablet a77984b7b08c43c6ba25f7426e7a3b3f (DEFAULT_TABLE table=default.Table_f13f6c5d2118406cb4485657e33bb75f [id=5d4c4cb185c54ea1a2dcb8c0dedb251a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.236272 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 732e887a691742f9ba454ed525403f61 (DEFAULT_TABLE table=default.Table_cb0c959c105d4fbc94aed34c2d82ef44 [id=fd1b88446fa34b8aba27bb21cf4d686b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.236338 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 12b4cf42b11a4560a9593232d7c2d5c2 (DEFAULT_TABLE table=default.Table_5b8939a4349f4b168b7fb7a1003050b8 [id=a8114b4158064620a08aef82c5346446]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.236472 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b5228d4131446f8460917e961b8f15. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.236661 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 225fdab102544960b0765bf82531ccf2 (DEFAULT_TABLE table=default.Table_e7879514239f4bf68e2e60480543af6f [id=c0c24dc28d324df9be49a179f70ccaad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.236877 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225fdab102544960b0765bf82531ccf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.237253 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87c0f1accc254ee19372bcff9a71ed5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.237294 8546 tablet_bootstrap.cc:492] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.237380 8546 ts_tablet_manager.cc:1403] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.001s
I20251028 03:00:00.237886 8546 raft_consensus.cc:359] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.237999 8546 raft_consensus.cc:385] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.238049 8546 raft_consensus.cc:740] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.238168 8546 consensus_queue.cc:260] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.238441 8546 ts_tablet_manager.cc:1434] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:00.238672 8546 tablet_bootstrap.cc:492] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.238981 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 585cb43a1daf4e90b45a1372a3f23450 (DEFAULT_TABLE table=default.Table_5eb75de4388641e4bffe66784f304fcc [id=639b9aeb45ea442480663cabd64de9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.239205 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 585cb43a1daf4e90b45a1372a3f23450. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.239698 8546 tablet_bootstrap.cc:654] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.240540 8384 tablet_service.cc:1505] Processing CreateTablet for tablet a77984b7b08c43c6ba25f7426e7a3b3f (DEFAULT_TABLE table=default.Table_f13f6c5d2118406cb4485657e33bb75f [id=5d4c4cb185c54ea1a2dcb8c0dedb251a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.240624 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 12b4cf42b11a4560a9593232d7c2d5c2 (DEFAULT_TABLE table=default.Table_5b8939a4349f4b168b7fb7a1003050b8 [id=a8114b4158064620a08aef82c5346446]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.240862 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b4cf42b11a4560a9593232d7c2d5c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.240834 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 732e887a691742f9ba454ed525403f61 (DEFAULT_TABLE table=default.Table_cb0c959c105d4fbc94aed34c2d82ef44 [id=fd1b88446fa34b8aba27bb21cf4d686b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.241060 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732e887a691742f9ba454ed525403f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.241227 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 225fdab102544960b0765bf82531ccf2 (DEFAULT_TABLE table=default.Table_e7879514239f4bf68e2e60480543af6f [id=c0c24dc28d324df9be49a179f70ccaad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.241356 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 36cfa812c550475385980578108fb6be (DEFAULT_TABLE table=default.Table_28717f0c640748cca2efe4a69c6b3a2b [id=5ca65c902c714f2da09d786f77854a74]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.241428 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225fdab102544960b0765bf82531ccf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.241551 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36cfa812c550475385980578108fb6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.241523 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 36cfa812c550475385980578108fb6be (DEFAULT_TABLE table=default.Table_28717f0c640748cca2efe4a69c6b3a2b [id=5ca65c902c714f2da09d786f77854a74]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.241719 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36cfa812c550475385980578108fb6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.243281 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 585cb43a1daf4e90b45a1372a3f23450 (DEFAULT_TABLE table=default.Table_5eb75de4388641e4bffe66784f304fcc [id=639b9aeb45ea442480663cabd64de9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.243484 8122 tablet_service.cc:1505] Processing CreateTablet for tablet efbbfee9d8c44e5489e7a24fdd46f290 (DEFAULT_TABLE table=default.Table_d28a01f0c50b410eb906392280a7144c [id=892f0865ce7a4e4a9c1164553925db70]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.243528 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 585cb43a1daf4e90b45a1372a3f23450. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.243680 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efbbfee9d8c44e5489e7a24fdd46f290. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.245239 8259 tablet_service.cc:1505] Processing CreateTablet for tablet a77984b7b08c43c6ba25f7426e7a3b3f (DEFAULT_TABLE table=default.Table_f13f6c5d2118406cb4485657e33bb75f [id=5d4c4cb185c54ea1a2dcb8c0dedb251a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.245342 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 12b4cf42b11a4560a9593232d7c2d5c2 (DEFAULT_TABLE table=default.Table_5b8939a4349f4b168b7fb7a1003050b8 [id=a8114b4158064620a08aef82c5346446]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.245479 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a77984b7b08c43c6ba25f7426e7a3b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.245677 8253 tablet_service.cc:1505] Processing CreateTablet for tablet efbbfee9d8c44e5489e7a24fdd46f290 (DEFAULT_TABLE table=default.Table_d28a01f0c50b410eb906392280a7144c [id=892f0865ce7a4e4a9c1164553925db70]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.245719 8388 tablet_service.cc:1505] Processing CreateTablet for tablet a279d68990284332bae453e4d26f1036 (DEFAULT_TABLE table=default.Table_a96f9bde439b4ee0a079bb871e47b3a8 [id=c6ecd003e8b24bfe914d7bd27f1fd41a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.245745 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 225fdab102544960b0765bf82531ccf2 (DEFAULT_TABLE table=default.Table_e7879514239f4bf68e2e60480543af6f [id=c0c24dc28d324df9be49a179f70ccaad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.245894 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efbbfee9d8c44e5489e7a24fdd46f290. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.245934 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a279d68990284332bae453e4d26f1036. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.246080 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 585cb43a1daf4e90b45a1372a3f23450 (DEFAULT_TABLE table=default.Table_5eb75de4388641e4bffe66784f304fcc [id=639b9aeb45ea442480663cabd64de9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.246279 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 585cb43a1daf4e90b45a1372a3f23450. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.247494 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 732e887a691742f9ba454ed525403f61 (DEFAULT_TABLE table=default.Table_cb0c959c105d4fbc94aed34c2d82ef44 [id=fd1b88446fa34b8aba27bb21cf4d686b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.247694 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732e887a691742f9ba454ed525403f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.247917 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b4cf42b11a4560a9593232d7c2d5c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.248427 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 04723e51ee66454182da19634c14ffb4 (DEFAULT_TABLE table=default.Table_008b1b0c028d4fdface45a3b2110508d [id=b8ae9b94370c4b3ba1ac2af86c4762d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.248469 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 04723e51ee66454182da19634c14ffb4 (DEFAULT_TABLE table=default.Table_008b1b0c028d4fdface45a3b2110508d [id=b8ae9b94370c4b3ba1ac2af86c4762d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.248649 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04723e51ee66454182da19634c14ffb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.248658 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04723e51ee66454182da19634c14ffb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.250407 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 36cfa812c550475385980578108fb6be (DEFAULT_TABLE table=default.Table_28717f0c640748cca2efe4a69c6b3a2b [id=5ca65c902c714f2da09d786f77854a74]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.236338 8115 tablet_service.cc:1505] Processing CreateTablet for tablet a279d68990284332bae453e4d26f1036 (DEFAULT_TABLE table=default.Table_a96f9bde439b4ee0a079bb871e47b3a8 [id=c6ecd003e8b24bfe914d7bd27f1fd41a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.250624 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36cfa812c550475385980578108fb6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.250833 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a279d68990284332bae453e4d26f1036. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.250972 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a77984b7b08c43c6ba25f7426e7a3b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.252729 8252 tablet_service.cc:1505] Processing CreateTablet for tablet a279d68990284332bae453e4d26f1036 (DEFAULT_TABLE table=default.Table_a96f9bde439b4ee0a079bb871e47b3a8 [id=c6ecd003e8b24bfe914d7bd27f1fd41a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.252935 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a279d68990284332bae453e4d26f1036. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.253895 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225fdab102544960b0765bf82531ccf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.256739 8400 tablet_service.cc:1505] Processing CreateTablet for tablet efbbfee9d8c44e5489e7a24fdd46f290 (DEFAULT_TABLE table=default.Table_d28a01f0c50b410eb906392280a7144c [id=892f0865ce7a4e4a9c1164553925db70]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.256994 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efbbfee9d8c44e5489e7a24fdd46f290. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.258440 8384 tablet_service.cc:1505] Processing CreateTablet for tablet ec804f90785041faafcc90c0dbece717 (DEFAULT_TABLE table=default.Table_173b483053f948f58173aadecf7e6765 [id=ae8c48e97de540fc88cc4682e717331c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.258733 8384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec804f90785041faafcc90c0dbece717. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.258867 8261 tablet_service.cc:1505] Processing CreateTablet for tablet fb59e5f7bb57488ba2f9e4b43a118bf8 (DEFAULT_TABLE table=default.Table_80603ad4c2bc4966bbc952208ce043e2 [id=a222e22ca83a46f793259accf7b4bfbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.259120 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb59e5f7bb57488ba2f9e4b43a118bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.260571 8544 tablet_bootstrap.cc:492] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.260675 8544 ts_tablet_manager.cc:1403] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.000s
I20251028 03:00:00.260759 8259 tablet_service.cc:1505] Processing CreateTablet for tablet ec804f90785041faafcc90c0dbece717 (DEFAULT_TABLE table=default.Table_173b483053f948f58173aadecf7e6765 [id=ae8c48e97de540fc88cc4682e717331c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.260965 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec804f90785041faafcc90c0dbece717. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.261217 8544 raft_consensus.cc:359] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.261334 8544 raft_consensus.cc:385] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.261370 8544 raft_consensus.cc:740] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.261490 8544 consensus_queue.cc:260] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.261803 8544 ts_tablet_manager.cc:1434] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.261835 8121 tablet_service.cc:1505] Processing CreateTablet for tablet ada2c7c883014aa597ad8dbe14bf51b3 (DEFAULT_TABLE table=default.Table_0325897e2eb5414685dd8be9e6d2fd2e [id=b9c19c3aa17e4059b3f58533143edff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.261866 8120 tablet_service.cc:1505] Processing CreateTablet for tablet ea7297980d07434da8676af4cd029ac2 (DEFAULT_TABLE table=default.Table_7f8968a27516426095ca46f368ba84b0 [id=92a0872e00394335812005ecc6b2de86]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.261988 8544 tablet_bootstrap.cc:492] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.262084 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada2c7c883014aa597ad8dbe14bf51b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.262214 8126 tablet_service.cc:1505] Processing CreateTablet for tablet 5693431412f247bea0e0913c9e5b90e9 (DEFAULT_TABLE table=default.Table_b7c2ed1bcbef411dbf047fbd023915b0 [id=3e6c65437a3547889a19d08050c294fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.262637 8112 tablet_service.cc:1505] Processing CreateTablet for tablet c5aa5e6d12eb4140aa803173cfae931e (DEFAULT_TABLE table=default.Table_55aa729834954e5cadb191355a8a754e [id=b6f0517e97ba461cb9f35f497f4a9bd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.263273 8544 tablet_bootstrap.cc:654] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.263374 8122 tablet_service.cc:1505] Processing CreateTablet for tablet ec804f90785041faafcc90c0dbece717 (DEFAULT_TABLE table=default.Table_173b483053f948f58173aadecf7e6765 [id=ae8c48e97de540fc88cc4682e717331c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.263612 8117 tablet_service.cc:1505] Processing CreateTablet for tablet dfda4b9d46ae427baede4b8b44c2a843 (DEFAULT_TABLE table=default.Table_c5aa8eae0ee64a58abe614181b524864 [id=bae2c0c5846d4cf69eeac13da536c4cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.263865 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 23510a769098491b8355b645e46b0096 (DEFAULT_TABLE table=default.Table_77bf3f2df56847a5988614b150aa5831 [id=2853d56f2a8e442393d5d15d98d760e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.264089 8115 tablet_service.cc:1505] Processing CreateTablet for tablet 04723e51ee66454182da19634c14ffb4 (DEFAULT_TABLE table=default.Table_008b1b0c028d4fdface45a3b2110508d [id=b8ae9b94370c4b3ba1ac2af86c4762d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.264891 8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5693431412f247bea0e0913c9e5b90e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.264962 8113 tablet_service.cc:1505] Processing CreateTablet for tablet bef124efe7934e5284441693573e7f40 (DEFAULT_TABLE table=default.Table_df4ca62b1a17423d8e824e6eb889a2c4 [id=e334dbdebde14bd992b9fb2f666e59ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.265161 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bef124efe7934e5284441693573e7f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.266469 8124 tablet_service.cc:1505] Processing CreateTablet for tablet b6e67d8a267b4f129131357c17387e4b (DEFAULT_TABLE table=default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.266551 8123 tablet_service.cc:1505] Processing CreateTablet for tablet b1b968c0866b43b7b1dd94e8d1a6c268 (DEFAULT_TABLE table=default.Table_d077de0ccc6448cf9de68fc6f1803ce4 [id=41581310d2e1464e8fbd23adc49ea092]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.266688 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e67d8a267b4f129131357c17387e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.266801 8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b968c0866b43b7b1dd94e8d1a6c268. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.266887 9839 raft_consensus.cc:493] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.266991 9839 raft_consensus.cc:515] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.267042 8114 tablet_service.cc:1505] Processing CreateTablet for tablet 566787825efe4f5aafd0d1f9442c8b26 (DEFAULT_TABLE table=default.Table_b6dbdbfa2618496c8858dabf12f02366 [id=042c6fa7c4434eff9244aedace40ec03]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.267232 8114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 566787825efe4f5aafd0d1f9442c8b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.267450 9839 leader_election.cc:290] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.268131 8128 tablet_service.cc:1505] Processing CreateTablet for tablet 7e7d519bd243449199f83dc0219169b9 (DEFAULT_TABLE table=default.Table_00a06d6bfdff457ab18a0404c944f780 [id=27cec05ebbf74601b28ed49f36637dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.268344 8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e7d519bd243449199f83dc0219169b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.270413 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5aa5e6d12eb4140aa803173cfae931e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.270663 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e95a45cb612447dd9bebd9f7a019d1f9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.270768 8143 raft_consensus.cc:2468] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:00.271075 8354 leader_election.cc:304] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:00.271247 9839 raft_consensus.cc:2804] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.271287 9839 raft_consensus.cc:493] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.271358 9839 raft_consensus.cc:3060] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.272470 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec804f90785041faafcc90c0dbece717. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.272629 9839 raft_consensus.cc:515] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.272917 8398 tablet_service.cc:1505] Processing CreateTablet for tablet fb59e5f7bb57488ba2f9e4b43a118bf8 (DEFAULT_TABLE table=default.Table_80603ad4c2bc4966bbc952208ce043e2 [id=a222e22ca83a46f793259accf7b4bfbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.273104 9839 leader_election.cc:290] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.273203 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb59e5f7bb57488ba2f9e4b43a118bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.273845 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e95a45cb612447dd9bebd9f7a019d1f9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.274008 8273 raft_consensus.cc:2468] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:00.274899 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfda4b9d46ae427baede4b8b44c2a843. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.274957 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e95a45cb612447dd9bebd9f7a019d1f9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:00.274952 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e95a45cb612447dd9bebd9f7a019d1f9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.275059 8273 raft_consensus.cc:3060] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.275059 8142 raft_consensus.cc:3060] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.276014 8142 raft_consensus.cc:2468] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.276064 8273 raft_consensus.cc:2468] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.276365 8356 leader_election.cc:304] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.276748 9639 raft_consensus.cc:2804] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.276825 9639 raft_consensus.cc:697] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:00.276957 9639 consensus_queue.cc:237] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.277204 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23510a769098491b8355b645e46b0096. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.279286 8544 tablet_bootstrap.cc:492] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.279419 8544 ts_tablet_manager.cc:1403] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20251028 03:00:00.280066 8544 raft_consensus.cc:359] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.280169 8544 raft_consensus.cc:385] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.280205 8544 raft_consensus.cc:740] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.280246 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04723e51ee66454182da19634c14ffb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.280319 8544 consensus_queue.cc:260] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.280627 8544 ts_tablet_manager.cc:1434] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:00.280808 8544 tablet_bootstrap.cc:492] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.280819 9840 raft_consensus.cc:493] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.280891 9840 raft_consensus.cc:515] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.281222 9840 leader_election.cc:290] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.281847 8544 tablet_bootstrap.cc:654] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.282061 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea7297980d07434da8676af4cd029ac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.282157 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "923e46aca8284966980ac5b062baf829" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.282279 8273 raft_consensus.cc:2468] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:00.283160 8544 tablet_bootstrap.cc:492] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.283246 8544 ts_tablet_manager.cc:1403] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20251028 03:00:00.283288 8356 leader_election.cc:304] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.283509 9840 raft_consensus.cc:2804] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.283566 9840 raft_consensus.cc:493] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.283627 9840 raft_consensus.cc:3060] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.283728 8544 raft_consensus.cc:359] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.283836 8544 raft_consensus.cc:385] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.283879 8544 raft_consensus.cc:740] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.283989 8544 consensus_queue.cc:260] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.284263 8544 ts_tablet_manager.cc:1434] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.284435 8544 tablet_bootstrap.cc:492] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.284541 9840 raft_consensus.cc:515] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.284879 9840 leader_election.cc:290] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.285511 8544 tablet_bootstrap.cc:654] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.286839 8252 tablet_service.cc:1505] Processing CreateTablet for tablet bef124efe7934e5284441693573e7f40 (DEFAULT_TABLE table=default.Table_df4ca62b1a17423d8e824e6eb889a2c4 [id=e334dbdebde14bd992b9fb2f666e59ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.286871 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "923e46aca8284966980ac5b062baf829" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.286969 8147 raft_consensus.cc:3060] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.287148 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bef124efe7934e5284441693573e7f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.287220 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "923e46aca8284966980ac5b062baf829" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.287134 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 23510a769098491b8355b645e46b0096 (DEFAULT_TABLE table=default.Table_77bf3f2df56847a5988614b150aa5831 [id=2853d56f2a8e442393d5d15d98d760e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.287992 8147 raft_consensus.cc:2468] T 923e46aca8284966980ac5b062baf829 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.288357 8257 tablet_service.cc:1505] Processing CreateTablet for tablet dfda4b9d46ae427baede4b8b44c2a843 (DEFAULT_TABLE table=default.Table_c5aa8eae0ee64a58abe614181b524864 [id=bae2c0c5846d4cf69eeac13da536c4cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.288569 9888 raft_consensus.cc:493] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.288597 8354 leader_election.cc:304] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:00.288646 9888 raft_consensus.cc:515] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.288765 9839 raft_consensus.cc:2804] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.288841 8544 tablet_bootstrap.cc:492] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.288849 9839 raft_consensus.cc:697] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:00.288913 8544 ts_tablet_manager.cc:1403] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 03:00:00.288970 9888 leader_election.cc:290] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.288973 9839 consensus_queue.cc:237] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.289234 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "923e46aca8284966980ac5b062baf829" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:00.289386 8273 raft_consensus.cc:3060] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.289390 8544 raft_consensus.cc:359] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.289479 8544 raft_consensus.cc:385] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.289522 8544 raft_consensus.cc:740] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.289680 8544 consensus_queue.cc:260] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.289886 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23510a769098491b8355b645e46b0096. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.290020 8544 ts_tablet_manager.cc:1434] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.290205 8544 tablet_bootstrap.cc:492] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.290594 8273 raft_consensus.cc:2468] T 923e46aca8284966980ac5b062baf829 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.291431 8544 tablet_bootstrap.cc:654] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.292169 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfda4b9d46ae427baede4b8b44c2a843. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.292320 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 23510a769098491b8355b645e46b0096 (DEFAULT_TABLE table=default.Table_77bf3f2df56847a5988614b150aa5831 [id=2853d56f2a8e442393d5d15d98d760e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.292572 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23510a769098491b8355b645e46b0096. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.293130 8388 tablet_service.cc:1505] Processing CreateTablet for tablet bef124efe7934e5284441693573e7f40 (DEFAULT_TABLE table=default.Table_df4ca62b1a17423d8e824e6eb889a2c4 [id=e334dbdebde14bd992b9fb2f666e59ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.293391 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bef124efe7934e5284441693573e7f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.292333 8400 tablet_service.cc:1505] Processing CreateTablet for tablet c5aa5e6d12eb4140aa803173cfae931e (DEFAULT_TABLE table=default.Table_55aa729834954e5cadb191355a8a754e [id=b6f0517e97ba461cb9f35f497f4a9bd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.293840 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5aa5e6d12eb4140aa803173cfae931e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.294819 8386 tablet_service.cc:1505] Processing CreateTablet for tablet dfda4b9d46ae427baede4b8b44c2a843 (DEFAULT_TABLE table=default.Table_c5aa8eae0ee64a58abe614181b524864 [id=bae2c0c5846d4cf69eeac13da536c4cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.295022 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfda4b9d46ae427baede4b8b44c2a843. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.295142 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "464e6b3483bc4816bfa844c52fd3ca53" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:00.295538 8083 leader_election.cc:343] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.295921 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "464e6b3483bc4816bfa844c52fd3ca53" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.296461 8083 leader_election.cc:343] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.296546 8083 leader_election.cc:304] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.296722 9888 raft_consensus.cc:2749] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.296948 8259 tablet_service.cc:1505] Processing CreateTablet for tablet c5aa5e6d12eb4140aa803173cfae931e (DEFAULT_TABLE table=default.Table_55aa729834954e5cadb191355a8a754e [id=b6f0517e97ba461cb9f35f497f4a9bd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.297206 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5aa5e6d12eb4140aa803173cfae931e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.298252 8544 tablet_bootstrap.cc:492] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.298350 8544 ts_tablet_manager.cc:1403] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20251028 03:00:00.298887 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_5ba8c01948c043be8f3f6f6112a40078 [id=6d4246a1b1ea4448b8fbf6a5abecfed1] alter complete (version 2)
I20251028 03:00:00.299082 8544 raft_consensus.cc:359] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.299217 8544 raft_consensus.cc:385] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.299250 8544 raft_consensus.cc:740] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.299402 8544 consensus_queue.cc:260] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.299727 8544 ts_tablet_manager.cc:1434] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.299917 8544 tablet_bootstrap.cc:492] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.301168 8544 tablet_bootstrap.cc:654] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.302553 8260 tablet_service.cc:1505] Processing CreateTablet for tablet b6e67d8a267b4f129131357c17387e4b (DEFAULT_TABLE table=default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.302649 8251 tablet_service.cc:1505] Processing CreateTablet for tablet b1b968c0866b43b7b1dd94e8d1a6c268 (DEFAULT_TABLE table=default.Table_d077de0ccc6448cf9de68fc6f1803ce4 [id=41581310d2e1464e8fbd23adc49ea092]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.302832 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e67d8a267b4f129131357c17387e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.303098 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b968c0866b43b7b1dd94e8d1a6c268. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.303810 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 5693431412f247bea0e0913c9e5b90e9 (DEFAULT_TABLE table=default.Table_b7c2ed1bcbef411dbf047fbd023915b0 [id=3e6c65437a3547889a19d08050c294fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.304032 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5693431412f247bea0e0913c9e5b90e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.306061 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet 8720ea8c68904d57a3fc5c0e94d338ca from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:00.306133 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 1c429d74609043818c89652f00a38a56 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:00.306593 8397 tablet_service.cc:1505] Processing CreateTablet for tablet ea7297980d07434da8676af4cd029ac2 (DEFAULT_TABLE table=default.Table_7f8968a27516426095ca46f368ba84b0 [id=92a0872e00394335812005ecc6b2de86]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.306658 9840 raft_consensus.cc:493] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.306742 9840 raft_consensus.cc:515] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.306895 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea7297980d07434da8676af4cd029ac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.307092 8389 tablet_service.cc:1505] Processing CreateTablet for tablet 5693431412f247bea0e0913c9e5b90e9 (DEFAULT_TABLE table=default.Table_b7c2ed1bcbef411dbf047fbd023915b0 [id=3e6c65437a3547889a19d08050c294fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.307356 8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5693431412f247bea0e0913c9e5b90e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.307139 9840 leader_election.cc:290] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.306593 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 7e7d519bd243449199f83dc0219169b9 (DEFAULT_TABLE table=default.Table_00a06d6bfdff457ab18a0404c944f780 [id=27cec05ebbf74601b28ed49f36637dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.306593 8385 tablet_service.cc:1505] Processing CreateTablet for tablet ada2c7c883014aa597ad8dbe14bf51b3 (DEFAULT_TABLE table=default.Table_0325897e2eb5414685dd8be9e6d2fd2e [id=b9c19c3aa17e4059b3f58533143edff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.308449 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada2c7c883014aa597ad8dbe14bf51b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.309511 8401 tablet_service.cc:1505] Processing CreateTablet for tablet b6e67d8a267b4f129131357c17387e4b (DEFAULT_TABLE table=default.Table_3a29fd732cf24fcdadc69d00d2175216 [id=599c0c567e774367abc573ea344e8fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.309756 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e67d8a267b4f129131357c17387e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.310366 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e7d519bd243449199f83dc0219169b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:00.311412 8354 leader_election.cc:343] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 55e72a7f8aa842049da7aeaaa7001a28
I20251028 03:00:00.311879 8390 tablet_service.cc:1505] Processing CreateTablet for tablet b1b968c0866b43b7b1dd94e8d1a6c268 (DEFAULT_TABLE table=default.Table_d077de0ccc6448cf9de68fc6f1803ce4 [id=41581310d2e1464e8fbd23adc49ea092]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.312148 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b968c0866b43b7b1dd94e8d1a6c268. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.312250 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 566787825efe4f5aafd0d1f9442c8b26 (DEFAULT_TABLE table=default.Table_b6dbdbfa2618496c8858dabf12f02366 [id=042c6fa7c4434eff9244aedace40ec03]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.312500 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 566787825efe4f5aafd0d1f9442c8b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.314138 8391 tablet_service.cc:1505] Processing CreateTablet for tablet 566787825efe4f5aafd0d1f9442c8b26 (DEFAULT_TABLE table=default.Table_b6dbdbfa2618496c8858dabf12f02366 [id=042c6fa7c4434eff9244aedace40ec03]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.314374 8391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 566787825efe4f5aafd0d1f9442c8b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.314610 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "55e72a7f8aa842049da7aeaaa7001a28" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.314877 8249 tablet_service.cc:1505] Processing CreateTablet for tablet ada2c7c883014aa597ad8dbe14bf51b3 (DEFAULT_TABLE table=default.Table_0325897e2eb5414685dd8be9e6d2fd2e [id=b9c19c3aa17e4059b3f58533143edff6]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 03:00:00.314950 8356 leader_election.cc:343] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.315025 8356 leader_election.cc:304] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:00.315131 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada2c7c883014aa597ad8dbe14bf51b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.315223 9840 raft_consensus.cc:2749] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.315481 8257 tablet_service.cc:1505] Processing CreateTablet for tablet ea7297980d07434da8676af4cd029ac2 (DEFAULT_TABLE table=default.Table_7f8968a27516426095ca46f368ba84b0 [id=92a0872e00394335812005ecc6b2de86]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.315698 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea7297980d07434da8676af4cd029ac2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.315877 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb59e5f7bb57488ba2f9e4b43a118bf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.316038 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a77984b7b08c43c6ba25f7426e7a3b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.316179 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b4cf42b11a4560a9593232d7c2d5c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.316310 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732e887a691742f9ba454ed525403f61. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.316628 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 55e72a7f8aa842049da7aeaaa7001a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.316872 8551 tablet_replica.cc:333] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.317018 8551 raft_consensus.cc:2243] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.317117 8551 raft_consensus.cc:2272] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.317621 9888 raft_consensus.cc:493] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.317691 9888 raft_consensus.cc:515] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.318078 9888 leader_election.cc:290] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.321411 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 3f04c867f0ce4ab99f980cca063abc30 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.323177 8551 ts_tablet_manager.cc:1916] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.324018 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 1979906599754f7ba049c7a9b23bf26d (table default.Table_cdab567cf8da46e8a4d7ba2a80e1e2d9 [id=9fcc70dd88724a28b404e9d4109015aa]). Expected version 1 got 0
I20251028 03:00:00.325240 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 7e7d519bd243449199f83dc0219169b9 (DEFAULT_TABLE table=default.Table_00a06d6bfdff457ab18a0404c944f780 [id=27cec05ebbf74601b28ed49f36637dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.325524 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa36c6870c1b4a459f0a41c84855a76e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.325600 8551 ts_tablet_manager.cc:1929] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.325580 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_94da0c3c823c4016b6534ddecabf64a9" } modify_external_catalogs: true
I20251028 03:00:00.325664 8551 log.cc:1199] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/55e72a7f8aa842049da7aeaaa7001a28
I20251028 03:00:00.325699 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_94da0c3c823c4016b6534ddecabf64a9" } modify_external_catalogs: true
I20251028 03:00:00.325906 8551 ts_tablet_manager.cc:1950] T 55e72a7f8aa842049da7aeaaa7001a28 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 03:00:00.326228 8083 leader_election.cc:343] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.326669 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e7d519bd243449199f83dc0219169b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.329178 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1023c48e40024553b0795a912f2fdd18" } modify_external_catalogs: true
I20251028 03:00:00.329205 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_9b47e45dc8f8444e8be57929ceef64d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.329300 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1023c48e40024553b0795a912f2fdd18" } modify_external_catalogs: true
W20251028 03:00:00.329576 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b47e45dc8f8444e8be57929ceef64d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.330483 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_356d26cd8d9744b290961390a0918f05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.330768 9913 tablet.cc:1722] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251028 03:00:00.330819 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_356d26cd8d9744b290961390a0918f05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.331498 9909 tablet.cc:1722] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:00.332055 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa36c6870c1b4a459f0a41c84855a76e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.332371 8083 leader_election.cc:343] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.332449 8083 leader_election.cc:304] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.333454 9888 raft_consensus.cc:2749] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.334247 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_86557fb2d1974f27bee360a4c8355497" } new_table_name: "default.Table_b560e3cf638a42efa393170985fc08eb" modify_external_catalogs: true
I20251028 03:00:00.334298 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5b8939a4349f4b168b7fb7a1003050b8" } new_table_name: "default.Table_9e9610f75ecb42a0ad013fff6a7be344" modify_external_catalogs: true
I20251028 03:00:00.334995 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_37a7da4dda72400d81691a3598906d38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.335259 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37a7da4dda72400d81691a3598906d38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.335487 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 05b6a671e514463bace6c3232700a110 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:00.336115 8390 tablet_service.cc:1505] Processing CreateTablet for tablet 01dab674843144d7aff3fb711d5c1cc7 (DEFAULT_TABLE table=default.Table_9032b689a6154ff8bfd6bb7c5cfc1373 [id=289efe4655504c11afd56929343a33d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.336406 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01dab674843144d7aff3fb711d5c1cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.336613 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 6a0ea69a0c3d45c4803c23787eae09ec (DEFAULT_TABLE table=default.Table_71f1dfc334c447899b2d951d85d4547a [id=b76fa9078add4245a655c9cca1d63cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.336828 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a0ea69a0c3d45c4803c23787eae09ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.338832 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 58a0aea887d745af90d6909700761607 (DEFAULT_TABLE table=default.Table_86557fb2d1974f27bee360a4c8355497 [id=8e7efd3084374978b609775e032e9f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.339051 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58a0aea887d745af90d6909700761607. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.340096 8401 tablet_service.cc:1505] Processing CreateTablet for tablet dea4de43bc384ac8bd55472aa1e59c7e (DEFAULT_TABLE table=default.Table_13dbc4fadcfd41c8945172798836fb62 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.340299 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea4de43bc384ac8bd55472aa1e59c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.340425 7962 catalog_manager.cc:5649] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:00.340633 7962 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 1979906599754f7ba049c7a9b23bf26d (table default.Table_cdab567cf8da46e8a4d7ba2a80e1e2d9 [id=9fcc70dd88724a28b404e9d4109015aa]). Expected version 1 got 0
I20251028 03:00:00.340767 7962 catalog_manager.cc:5649] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:00.340924 7962 catalog_manager.cc:5649] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 03:00:00.341046 7962 catalog_manager.cc:5649] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:00.342221 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 58a0aea887d745af90d6909700761607 (DEFAULT_TABLE table=default.Table_86557fb2d1974f27bee360a4c8355497 [id=8e7efd3084374978b609775e032e9f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.342439 8263 tablet_service.cc:1505] Processing CreateTablet for tablet dea4de43bc384ac8bd55472aa1e59c7e (DEFAULT_TABLE table=default.Table_13dbc4fadcfd41c8945172798836fb62 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.342473 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 81c2c40e4da242f8891687a6365d273c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.342473 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58a0aea887d745af90d6909700761607. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.342646 8552 tablet_replica.cc:333] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.342796 8552 raft_consensus.cc:2243] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.342854 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a with delete_type TABLET_DATA_DELETED (replaced by tablet 4eaa472029a94052abb49241b7622d0f) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.342919 8552 raft_consensus.cc:2272] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.342814 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea4de43bc384ac8bd55472aa1e59c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.343499 8552 ts_tablet_manager.cc:1916] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:00.344960 7955 catalog_manager.cc:4952] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 3f04c867f0ce4ab99f980cca063abc30 because the tablet was not found. No further retry: Not found: Tablet not found: 3f04c867f0ce4ab99f980cca063abc30
I20251028 03:00:00.345525 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 55e72a7f8aa842049da7aeaaa7001a28 (table default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]) successfully deleted
I20251028 03:00:00.346289 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 01dab674843144d7aff3fb711d5c1cc7 (DEFAULT_TABLE table=default.Table_9032b689a6154ff8bfd6bb7c5cfc1373 [id=289efe4655504c11afd56929343a33d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.346606 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01dab674843144d7aff3fb711d5c1cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.347772 8552 ts_tablet_manager.cc:1929] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.347846 8552 log.cc:1199] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/81c2c40e4da242f8891687a6365d273c
I20251028 03:00:00.347905 8545 tablet_bootstrap.cc:492] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.348008 8545 ts_tablet_manager.cc:1403] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.116s user 0.002s sys 0.000s
I20251028 03:00:00.348075 8552 ts_tablet_manager.cc:1950] T 81c2c40e4da242f8891687a6365d273c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.348629 8552 tablet_replica.cc:333] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.348577 8545 raft_consensus.cc:359] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.348686 8545 raft_consensus.cc:385] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.348719 8545 raft_consensus.cc:740] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.348765 8552 raft_consensus.cc:2243] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.348856 8552 raft_consensus.cc:2272] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.348842 8545 consensus_queue.cc:260] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.348953 8544 tablet_bootstrap.cc:492] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.349033 8544 ts_tablet_manager.cc:1403] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.049s user 0.000s sys 0.002s
I20251028 03:00:00.349151 8545 ts_tablet_manager.cc:1434] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.349510 8544 raft_consensus.cc:359] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.349642 8544 raft_consensus.cc:385] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.349684 8544 raft_consensus.cc:740] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.349840 8544 consensus_queue.cc:260] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.350014 8552 ts_tablet_manager.cc:1916] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.350169 8544 ts_tablet_manager.cc:1434] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.350350 8544 tablet_bootstrap.cc:492] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.350908 9906 tablet.cc:1722] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:00.351326 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 81c2c40e4da242f8891687a6365d273c (table default.Table_5eddcfdf60ef439397a4948da2d85f90 [id=1cf0d713d3de40858825646f084e8505]) successfully deleted
I20251028 03:00:00.351576 8545 tablet_bootstrap.cc:492] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.352547 8545 tablet_bootstrap.cc:654] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.353333 8544 tablet_bootstrap.cc:654] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.353742 8552 ts_tablet_manager.cc:1929] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.353884 8552 log.cc:1199] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/1d5fcc9b539b46458cd8cbfc30cbcd0a
I20251028 03:00:00.354173 8552 ts_tablet_manager.cc:1950] T 1d5fcc9b539b46458cd8cbfc30cbcd0a P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.354625 8544 tablet_bootstrap.cc:492] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.354712 8544 ts_tablet_manager.cc:1403] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20251028 03:00:00.354959 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 1d5fcc9b539b46458cd8cbfc30cbcd0a (table default.Table_059639c7191541caa3dbb30b333d579f [id=5581f43ea3854dbeba821353ffce6ec7]) successfully deleted
I20251028 03:00:00.355201 8544 raft_consensus.cc:359] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.356005 8544 raft_consensus.cc:385] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.356074 8544 raft_consensus.cc:740] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.356062 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 01dab674843144d7aff3fb711d5c1cc7 (DEFAULT_TABLE table=default.Table_9032b689a6154ff8bfd6bb7c5cfc1373 [id=289efe4655504c11afd56929343a33d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.356262 8544 consensus_queue.cc:260] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.356352 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01dab674843144d7aff3fb711d5c1cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.356480 8545 tablet_bootstrap.cc:492] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.356563 8545 ts_tablet_manager.cc:1403] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 03:00:00.356564 8544 ts_tablet_manager.cc:1434] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 03:00:00.356976 8544 tablet_bootstrap.cc:492] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.357023 8545 raft_consensus.cc:359] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.357112 8545 raft_consensus.cc:385] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.357143 8545 raft_consensus.cc:740] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.357262 8545 consensus_queue.cc:260] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.357553 8545 ts_tablet_manager.cc:1434] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:00.357717 8545 tablet_bootstrap.cc:492] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.357975 8544 tablet_bootstrap.cc:654] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.358600 9888 raft_consensus.cc:493] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.358736 8545 tablet_bootstrap.cc:654] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.358683 9888 raft_consensus.cc:515] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.359079 9888 leader_election.cc:290] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.359385 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 58a0aea887d745af90d6909700761607 (DEFAULT_TABLE table=default.Table_86557fb2d1974f27bee360a4c8355497 [id=8e7efd3084374978b609775e032e9f87]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.359412 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4125bb9e525c4e7d91ae0862abc44584" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.359607 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58a0aea887d745af90d6909700761607. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.359702 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4125bb9e525c4e7d91ae0862abc44584" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:00.359807 8083 leader_election.cc:343] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.359830 8416 raft_consensus.cc:2468] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:00.360183 8083 leader_election.cc:304] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:00.360297 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 6a0ea69a0c3d45c4803c23787eae09ec (DEFAULT_TABLE table=default.Table_71f1dfc334c447899b2d951d85d4547a [id=b76fa9078add4245a655c9cca1d63cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.360505 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a0ea69a0c3d45c4803c23787eae09ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.360936 8545 tablet_bootstrap.cc:492] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.361016 8545 ts_tablet_manager.cc:1403] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 03:00:00.361471 8545 raft_consensus.cc:359] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.361562 8545 raft_consensus.cc:385] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.361591 8545 raft_consensus.cc:740] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.361732 8545 consensus_queue.cc:260] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.361896 9888 raft_consensus.cc:2804] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.361944 9888 raft_consensus.cc:493] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.361981 9888 raft_consensus.cc:3060] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.362149 8545 ts_tablet_manager.cc:1434] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.362306 8545 tablet_bootstrap.cc:492] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.362907 9888 raft_consensus.cc:515] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.363334 8116 tablet_service.cc:1505] Processing CreateTablet for tablet dea4de43bc384ac8bd55472aa1e59c7e (DEFAULT_TABLE table=default.Table_13dbc4fadcfd41c8945172798836fb62 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.363557 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea4de43bc384ac8bd55472aa1e59c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.363721 9888 leader_election.cc:290] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.363907 8545 tablet_bootstrap.cc:654] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.364058 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4125bb9e525c4e7d91ae0862abc44584" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:00.364383 8083 leader_election.cc:343] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.364892 8544 tablet_bootstrap.cc:492] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.364969 8544 ts_tablet_manager.cc:1403] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20251028 03:00:00.365399 8544 raft_consensus.cc:359] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.365483 8544 raft_consensus.cc:385] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.365512 8544 raft_consensus.cc:740] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.365618 8544 consensus_queue.cc:260] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.365635 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4125bb9e525c4e7d91ae0862abc44584" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:00.365733 8416 raft_consensus.cc:3060] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.365921 8544 ts_tablet_manager.cc:1434] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.366096 8544 tablet_bootstrap.cc:492] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.366155 8545 tablet_bootstrap.cc:492] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.366217 8545 ts_tablet_manager.cc:1403] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 03:00:00.366683 8416 raft_consensus.cc:2468] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:00.366655 8545 raft_consensus.cc:359] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.366802 8545 raft_consensus.cc:385] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.366840 8545 raft_consensus.cc:740] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.366969 8083 leader_election.cc:304] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:00.367081 8544 tablet_bootstrap.cc:654] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.367208 9788 raft_consensus.cc:2804] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.367278 9788 raft_consensus.cc:697] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:00.367439 9788 consensus_queue.cc:237] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.366979 8545 consensus_queue.cc:260] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.368531 8544 tablet_bootstrap.cc:492] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.368640 8544 ts_tablet_manager.cc:1403] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 03:00:00.369055 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet c120a3d56f6b41c28c7f7dcd84359a60 (table default.Table_13dbc4fadcfd41c8945172798836fb62 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]) successfully deleted
I20251028 03:00:00.369097 8544 raft_consensus.cc:359] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.369257 8544 raft_consensus.cc:385] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.369318 8544 raft_consensus.cc:740] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.369472 8544 consensus_queue.cc:260] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.369910 8544 ts_tablet_manager.cc:1434] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.370113 8544 tablet_bootstrap.cc:492] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.371248 8544 tablet_bootstrap.cc:654] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.371644 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 6a0ea69a0c3d45c4803c23787eae09ec (DEFAULT_TABLE table=default.Table_71f1dfc334c447899b2d951d85d4547a [id=b76fa9078add4245a655c9cca1d63cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.371951 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a0ea69a0c3d45c4803c23787eae09ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.372584 8544 tablet_bootstrap.cc:492] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.372660 8544 ts_tablet_manager.cc:1403] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 03:00:00.372717 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1c429d74609043818c89652f00a38a56
I20251028 03:00:00.373049 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 1c429d74609043818c89652f00a38a56 deleted and replaced by tablet 225ef744f6724f158981e051f2efbb43
I20251028 03:00:00.373097 8544 raft_consensus.cc:359] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.373183 8544 raft_consensus.cc:385] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.373207 8544 raft_consensus.cc:740] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.373312 8544 consensus_queue.cc:260] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.373574 8544 ts_tablet_manager.cc:1434] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.373817 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c429d74609043818c89652f00a38a56 with delete_type TABLET_DATA_DELETED (replaced by tablet 225ef744f6724f158981e051f2efbb43) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:00.374642 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 1c429d74609043818c89652f00a38a56 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c429d74609043818c89652f00a38a56 already in progress: creating tablet
I20251028 03:00:00.374979 8544 tablet_bootstrap.cc:492] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.376012 8544 tablet_bootstrap.cc:654] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.378340 8545 ts_tablet_manager.cc:1434] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20251028 03:00:00.378629 8545 tablet_bootstrap.cc:492] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.379660 8545 tablet_bootstrap.cc:654] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.381449 8545 tablet_bootstrap.cc:492] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.381541 8545 ts_tablet_manager.cc:1403] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 03:00:00.381991 8545 raft_consensus.cc:359] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.382133 8545 raft_consensus.cc:385] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.382170 8545 raft_consensus.cc:740] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.382328 8545 consensus_queue.cc:260] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.382709 8545 ts_tablet_manager.cc:1434] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.382886 8545 tablet_bootstrap.cc:492] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.383882 8545 tablet_bootstrap.cc:654] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.383965 8546 tablet_bootstrap.cc:492] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.384079 8546 ts_tablet_manager.cc:1403] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.145s user 0.002s sys 0.000s
I20251028 03:00:00.384851 8546 raft_consensus.cc:359] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.384970 8546 raft_consensus.cc:385] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.385005 8546 raft_consensus.cc:740] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.385161 8546 consensus_queue.cc:260] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.386133 8545 tablet_bootstrap.cc:492] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.386209 8545 ts_tablet_manager.cc:1403] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 03:00:00.387130 8545 raft_consensus.cc:359] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.387182 9840 raft_consensus.cc:493] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.387240 8545 raft_consensus.cc:385] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.387277 8545 raft_consensus.cc:740] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.387264 9840 raft_consensus.cc:515] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.387476 8545 consensus_queue.cc:260] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.387662 9840 leader_election.cc:290] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.387809 8545 ts_tablet_manager.cc:1434] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:00.388005 8545 tablet_bootstrap.cc:492] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.388155 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b436561f0bf44e0b2c680ce30919318" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.388295 8147 raft_consensus.cc:2468] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:00.388324 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b436561f0bf44e0b2c680ce30919318" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.388444 8273 raft_consensus.cc:2468] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:00.388598 8354 leader_election.cc:304] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:00.388773 9839 raft_consensus.cc:2804] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.388835 9839 raft_consensus.cc:493] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.388871 9839 raft_consensus.cc:3060] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.389293 8545 tablet_bootstrap.cc:654] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.390200 9839 raft_consensus.cc:515] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.390758 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b436561f0bf44e0b2c680ce30919318" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.390869 8147 raft_consensus.cc:3060] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.390903 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b436561f0bf44e0b2c680ce30919318" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:00.391008 8273 raft_consensus.cc:3060] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.391903 8147 raft_consensus.cc:2468] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.392095 8273 raft_consensus.cc:2468] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.392374 8354 leader_election.cc:304] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:00.392553 9839 leader_election.cc:290] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.392743 9839 raft_consensus.cc:2804] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.392931 9839 raft_consensus.cc:697] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:00.393107 9788 raft_consensus.cc:493] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.393205 9788 raft_consensus.cc:515] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.393565 9788 leader_election.cc:290] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:00.393649 8546 ts_tablet_manager.cc:1434] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20251028 03:00:00.393883 8546 tablet_bootstrap.cc:492] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.394344 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8b4b0aa30194ba495adca9138e2454e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.394690 8083 leader_election.cc:343] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.395026 8546 tablet_bootstrap.cc:654] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.395284 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8b4b0aa30194ba495adca9138e2454e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:00.396448 8083 leader_election.cc:343] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.396517 8083 leader_election.cc:304] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.396703 9788 raft_consensus.cc:2749] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.393105 9839 consensus_queue.cc:237] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:00.396919 7955 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 69e0358b593f4b55b6b4ec859e2c021d (table default.Table_49f2ad3b1c934eaeab556927604ddf1b [id=e88ffc49924340aa92a3aedd631330bf], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 69e0358b593f4b55b6b4ec859e2c021d consensus state has no leader
I20251028 03:00:00.398412 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c429d74609043818c89652f00a38a56 with delete_type TABLET_DATA_DELETED (replaced by tablet 225ef744f6724f158981e051f2efbb43) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:00.399674 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1c429d74609043818c89652f00a38a56 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c429d74609043818c89652f00a38a56 already in progress: creating tablet
I20251028 03:00:00.400058 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c429d74609043818c89652f00a38a56 with delete_type TABLET_DATA_DELETED (replaced by tablet 225ef744f6724f158981e051f2efbb43) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:00.401383 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 1c429d74609043818c89652f00a38a56 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1c429d74609043818c89652f00a38a56 already in progress: creating tablet
I20251028 03:00:00.403039 8546 tablet_bootstrap.cc:492] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.403177 8546 ts_tablet_manager.cc:1403] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 03:00:00.403230 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 97c0ebd67d0141ee9c823602213d0b2e
I20251028 03:00:00.403813 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b0ef051366704663b4e079ccdda63a43" } modify_external_catalogs: true
I20251028 03:00:00.403961 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 97c0ebd67d0141ee9c823602213d0b2e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.404256 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b0ef051366704663b4e079ccdda63a43" } modify_external_catalogs: true
W20251028 03:00:00.404397 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 97c0ebd67d0141ee9c823602213d0b2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97c0ebd67d0141ee9c823602213d0b2e already in progress: creating tablet
I20251028 03:00:00.405077 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet aa36c6870c1b4a459f0a41c84855a76e (table default.Table_d51065c1d57c4857bc5765d1baf9da06 [id=3815e9736bd34d668ec7f7e113e11e6f]). Expected version 1 got 0
I20251028 03:00:00.405268 8546 raft_consensus.cc:359] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.405377 8546 raft_consensus.cc:385] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.405411 8546 raft_consensus.cc:740] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.405639 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet 97c0ebd67d0141ee9c823602213d0b2e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.405805 8546 consensus_queue.cc:260] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.406009 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c6a86fd3f7c140eb852bd8a9bd48f8ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.406106 8546 ts_tablet_manager.cc:1434] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
W20251028 03:00:00.406256 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 97c0ebd67d0141ee9c823602213d0b2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97c0ebd67d0141ee9c823602213d0b2e already in progress: creating tablet
W20251028 03:00:00.406426 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6a86fd3f7c140eb852bd8a9bd48f8ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.407044 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f57b4fa3ceec4557bc6d6a2125d80790
I20251028 03:00:00.408102 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet f57b4fa3ceec4557bc6d6a2125d80790 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.408118 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6d5acf2d672a48d3bc24b9bd2d20a174"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.408265 8551 tablet_replica.cc:333] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.408386 8551 raft_consensus.cc:2243] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
W20251028 03:00:00.408432 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d5acf2d672a48d3bc24b9bd2d20a174 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.408607 8551 raft_consensus.cc:2272] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.408721 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet f57b4fa3ceec4557bc6d6a2125d80790 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.408876 8552 tablet_replica.cc:333] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.408917 8546 tablet_bootstrap.cc:492] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.409005 8552 raft_consensus.cc:2243] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.409062 8551 ts_tablet_manager.cc:1916] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.409111 8552 raft_consensus.cc:2272] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:00.409183 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 58a0aea887d745af90d6909700761607 consensus state has no leader
I20251028 03:00:00.409456 8552 ts_tablet_manager.cc:1916] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.409443 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a90dbf2ca45f4e3d8e9352c7eb8b3d99" } new_table_name: "default.Table_c7534bd73076465c96d28ee8be7d5800" modify_external_catalogs: true
W20251028 03:00:00.410194 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 12b4cf42b11a4560a9593232d7c2d5c2 consensus state has no leader
I20251028 03:00:00.410429 8546 tablet_bootstrap.cc:654] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.410398 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e529e86b94f949c8a4156848587444fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.410702 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e529e86b94f949c8a4156848587444fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.411473 8546 tablet_bootstrap.cc:492] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.411551 8546 ts_tablet_manager.cc:1403] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 03:00:00.411576 8551 ts_tablet_manager.cc:1929] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:00.411641 8551 log.cc:1199] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/f57b4fa3ceec4557bc6d6a2125d80790
I20251028 03:00:00.411880 8552 ts_tablet_manager.cc:1929] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:00.411908 8551 ts_tablet_manager.cc:1950] T f57b4fa3ceec4557bc6d6a2125d80790 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.411937 8552 log.cc:1199] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/f57b4fa3ceec4557bc6d6a2125d80790
I20251028 03:00:00.412087 8546 raft_consensus.cc:359] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.412091 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_aaeb21a72b4545c090ea639e8630a4d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.412197 8546 raft_consensus.cc:385] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.412245 8546 raft_consensus.cc:740] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.412359 8546 consensus_queue.cc:260] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:00.412391 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aaeb21a72b4545c090ea639e8630a4d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.412204 8552 ts_tablet_manager.cc:1950] T f57b4fa3ceec4557bc6d6a2125d80790 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.412775 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet f57b4fa3ceec4557bc6d6a2125d80790 (table default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]) successfully deleted
I20251028 03:00:00.413072 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 05b6a671e514463bace6c3232700a110
I20251028 03:00:00.413322 7965 catalog_manager.cc:6719] ReplaceTablet: tablet 05b6a671e514463bace6c3232700a110 deleted and replaced by tablet 2851b9c52e144c85a5f6673a8723c289
I20251028 03:00:00.413430 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet f57b4fa3ceec4557bc6d6a2125d80790 (table default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]) successfully deleted
I20251028 03:00:00.413477 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a387569232e54415a0dd77e906ddcee4" } modify_external_catalogs: true
I20251028 03:00:00.412637 8546 ts_tablet_manager.cc:1434] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.413561 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a387569232e54415a0dd77e906ddcee4" } modify_external_catalogs: true
I20251028 03:00:00.413763 8546 tablet_bootstrap.cc:492] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.413805 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 05b6a671e514463bace6c3232700a110 with delete_type TABLET_DATA_DELETED (replaced by tablet 2851b9c52e144c85a5f6673a8723c289) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:00.414201 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 05b6a671e514463bace6c3232700a110 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05b6a671e514463bace6c3232700a110 already in progress: creating tablet
I20251028 03:00:00.415124 8546 tablet_bootstrap.cc:654] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.413821 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 302e0ed455854eb2b661afc2f2b6921d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.416260 8551 tablet_replica.cc:333] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.416400 8551 raft_consensus.cc:2243] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.416508 8551 raft_consensus.cc:2272] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.416728 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_758c1adf3eb6460d9fa4e675a42507e3" } new_table_name: "default.Table_e72dcea543a9406bbf87d5de6e5fd9be" modify_external_catalogs: true
I20251028 03:00:00.416934 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet 05b6a671e514463bace6c3232700a110 with delete_type TABLET_DATA_DELETED (replaced by tablet 2851b9c52e144c85a5f6673a8723c289) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.416988 8551 ts_tablet_manager.cc:1916] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.417397 9958 raft_consensus.cc:493] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.417469 9958 raft_consensus.cc:515] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.417862 9958 leader_election.cc:290] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:00.418375 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d28a01f0c50b410eb906392280a7144c" } new_table_name: "default.Table_1a104ff3307b4f44bc0ee3bc31e8bfe7" modify_external_catalogs: true
W20251028 03:00:00.418666 8220 leader_election.cc:343] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 55e72a7f8aa842049da7aeaaa7001a28
W20251028 03:00:00.419096 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 05b6a671e514463bace6c3232700a110 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05b6a671e514463bace6c3232700a110 already in progress: creating tablet
I20251028 03:00:00.419375 9788 raft_consensus.cc:493] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.419489 9788 raft_consensus.cc:515] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.419559 8551 ts_tablet_manager.cc:1929] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.419624 8551 log.cc:1199] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/302e0ed455854eb2b661afc2f2b6921d
I20251028 03:00:00.419839 8551 ts_tablet_manager.cc:1950] T 302e0ed455854eb2b661afc2f2b6921d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.419867 9788 leader_election.cc:290] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:00.420382 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d3e422c7ae64b25960714ecc926046a" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.420445 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 302e0ed455854eb2b661afc2f2b6921d (table default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]) successfully deleted
W20251028 03:00:00.420848 8083 leader_election.cc:343] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.421180 8544 tablet_bootstrap.cc:492] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.421223 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d3e422c7ae64b25960714ecc926046a" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.421278 8544 ts_tablet_manager.cc:1403] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.000s
W20251028 03:00:00.421643 8083 leader_election.cc:343] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.421710 8083 leader_election.cc:304] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.421764 8544 raft_consensus.cc:359] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.421875 8544 raft_consensus.cc:385] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.421911 8544 raft_consensus.cc:740] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.421994 9788 raft_consensus.cc:2749] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.422036 8544 consensus_queue.cc:260] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.422353 8544 ts_tablet_manager.cc:1434] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.422844 8544 tablet_bootstrap.cc:492] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.424257 8544 tablet_bootstrap.cc:654] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.424603 9788 raft_consensus.cc:493] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.424687 9788 raft_consensus.cc:515] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.425009 9788 leader_election.cc:290] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:00.425343 8218 leader_election.cc:343] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 55e72a7f8aa842049da7aeaaa7001a28
I20251028 03:00:00.425419 8218 leader_election.cc:304] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:00.425611 9958 raft_consensus.cc:2749] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.426153 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bf9795bd59b42ea997a97c84e1ea111" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.426693 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bf9795bd59b42ea997a97c84e1ea111" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.427196 8083 leader_election.cc:343] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:00.427325 8083 leader_election.cc:343] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.427371 8083 leader_election.cc:304] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.427507 9888 raft_consensus.cc:493] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.427592 9888 raft_consensus.cc:515] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.427779 9788 raft_consensus.cc:2749] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.427919 9888 leader_election.cc:290] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.428627 8545 tablet_bootstrap.cc:492] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.428725 8545 ts_tablet_manager.cc:1403] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20251028 03:00:00.429236 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 97c0ebd67d0141ee9c823602213d0b2e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.429240 8545 raft_consensus.cc:359] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.429337 8545 raft_consensus.cc:385] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.429369 8545 raft_consensus.cc:740] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.429383 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 06f7d43bea27429fb45413e0dc0ba269 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.429484 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet a9653fc3236f4e0b812de5a859932587 with delete_type TABLET_DATA_DELETED (replaced by tablet 58268ac590f3437a942dcafd927a5496) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.429539 8545 consensus_queue.cc:260] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.429876 8545 ts_tablet_manager.cc:1434] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.430086 8545 tablet_bootstrap.cc:492] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.430318 8550 tablet_replica.cc:333] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.430461 8550 raft_consensus.cc:2243] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.430560 8550 raft_consensus.cc:2272] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.430764 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "593ac7984e01478184af49e5b87ceec4" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.431125 8545 tablet_bootstrap.cc:654] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:00.431128 8083 leader_election.cc:343] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.431581 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "593ac7984e01478184af49e5b87ceec4" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.431665 8550 ts_tablet_manager.cc:1916] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:00.433630 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 97c0ebd67d0141ee9c823602213d0b2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97c0ebd67d0141ee9c823602213d0b2e already in progress: creating tablet
W20251028 03:00:00.433772 8083 leader_election.cc:343] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.433856 8083 leader_election.cc:304] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.434053 8550 ts_tablet_manager.cc:1929] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.434129 8550 log.cc:1199] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/06f7d43bea27429fb45413e0dc0ba269
I20251028 03:00:00.434111 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet f57b4fa3ceec4557bc6d6a2125d80790 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.434223 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 05b6a671e514463bace6c3232700a110 with delete_type TABLET_DATA_DELETED (replaced by tablet 2851b9c52e144c85a5f6673a8723c289) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.434327 8550 ts_tablet_manager.cc:1950] T 06f7d43bea27429fb45413e0dc0ba269 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.434453 9788 raft_consensus.cc:2749] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.434625 8550 tablet_replica.cc:333] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.434746 8550 raft_consensus.cc:2243] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.434840 8550 raft_consensus.cc:2272] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.437373 7960 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706] alter complete (version 1)
W20251028 03:00:00.438167 9912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251028 03:00:00.438475 9912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251028 03:00:00.438536 9912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251028 03:00:00.439559 8546 tablet_bootstrap.cc:492] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.439551 8550 ts_tablet_manager.cc:1916] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.439664 8546 ts_tablet_manager.cc:1403] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20251028 03:00:00.440258 8546 raft_consensus.cc:359] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.440383 8546 raft_consensus.cc:385] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.440418 8546 raft_consensus.cc:740] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.440527 8546 consensus_queue.cc:260] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.440801 8546 ts_tablet_manager.cc:1434] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.441001 8546 tablet_bootstrap.cc:492] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.442016 8546 tablet_bootstrap.cc:654] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.442590 8550 ts_tablet_manager.cc:1929] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.442724 8550 log.cc:1199] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a9653fc3236f4e0b812de5a859932587
I20251028 03:00:00.443150 8550 ts_tablet_manager.cc:1950] T a9653fc3236f4e0b812de5a859932587 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.443665 8550 tablet_replica.cc:333] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.444043 8550 raft_consensus.cc:2243] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.444221 8550 raft_consensus.cc:2272] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.444312 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 06f7d43bea27429fb45413e0dc0ba269 (table default.Table_b001b0dab18e4d0a8a4421ddf1f836d9 [id=ee6840ec2cd44d6883833d367999752e]) successfully deleted
I20251028 03:00:00.444489 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a9653fc3236f4e0b812de5a859932587 (table default.Table_0b7b98e5884a48de833ab1fbbad97f08 [id=fdb289a561dd4c6497173ef43e2e9ccd]) successfully deleted
I20251028 03:00:00.445643 8546 tablet_bootstrap.cc:492] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.445729 8546 ts_tablet_manager.cc:1403] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 03:00:00.446231 8546 raft_consensus.cc:359] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.446336 8546 raft_consensus.cc:385] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.446367 8546 raft_consensus.cc:740] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.446470 8546 consensus_queue.cc:260] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.446568 8545 tablet_bootstrap.cc:492] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.446656 8545 ts_tablet_manager.cc:1403] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20251028 03:00:00.446750 8546 ts_tablet_manager.cc:1434] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.446938 8546 tablet_bootstrap.cc:492] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.447160 8545 raft_consensus.cc:359] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.447266 8545 raft_consensus.cc:385] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.447329 8545 raft_consensus.cc:740] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.447465 8545 consensus_queue.cc:260] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.447753 8545 ts_tablet_manager.cc:1434] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.447974 8545 tablet_bootstrap.cc:492] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.448035 8546 tablet_bootstrap.cc:654] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.448233 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2a2ebe28dbd2439e963319ec6e515635
I20251028 03:00:00.449357 8545 tablet_bootstrap.cc:654] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.450160 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 2a2ebe28dbd2439e963319ec6e515635 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.450613 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet 2a2ebe28dbd2439e963319ec6e515635 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.450675 9839 raft_consensus.cc:493] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.450737 8551 tablet_replica.cc:333] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.450832 8552 tablet_replica.cc:333] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.450902 8551 raft_consensus.cc:2243] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:00.450946 8552 raft_consensus.cc:2243] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.451043 8552 raft_consensus.cc:2272] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.451071 8551 raft_consensus.cc:2272] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.450750 9839 raft_consensus.cc:515] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.451447 8551 ts_tablet_manager.cc:1916] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.451717 9839 leader_election.cc:290] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.452131 8552 ts_tablet_manager.cc:1916] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.452572 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f28ca2cf41564792b49940aebc9b2c98" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.452680 8273 raft_consensus.cc:2468] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:00.453896 8551 ts_tablet_manager.cc:1929] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:00.453944 8546 tablet_bootstrap.cc:492] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.454020 8546 ts_tablet_manager.cc:1403] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 03:00:00.454028 8551 log.cc:1199] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2a2ebe28dbd2439e963319ec6e515635
I20251028 03:00:00.454298 8551 ts_tablet_manager.cc:1950] T 2a2ebe28dbd2439e963319ec6e515635 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.454512 8552 ts_tablet_manager.cc:1929] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:00.454576 8552 log.cc:1199] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/2a2ebe28dbd2439e963319ec6e515635
I20251028 03:00:00.454546 8546 raft_consensus.cc:359] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.454653 8546 raft_consensus.cc:385] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.454689 8546 raft_consensus.cc:740] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.454843 8552 ts_tablet_manager.cc:1950] T 2a2ebe28dbd2439e963319ec6e515635 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.454809 8546 consensus_queue.cc:260] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.455279 8546 ts_tablet_manager.cc:1434] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.455920 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 302e0ed455854eb2b661afc2f2b6921d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.455966 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet 55e72a7f8aa842049da7aeaaa7001a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.456135 8546 tablet_bootstrap.cc:492] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.456328 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 2a2ebe28dbd2439e963319ec6e515635 (table default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]) successfully deleted
I20251028 03:00:00.456434 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet d39008ac6324451a9d1ed54a2ccc3abf with delete_type TABLET_DATA_DELETED (replaced by tablet 985a0daa0a544216bbb06b1bc9d19231) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.456466 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2a2ebe28dbd2439e963319ec6e515635 (table default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]) successfully deleted
I20251028 03:00:00.456579 8544 tablet_bootstrap.cc:492] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.456665 8544 ts_tablet_manager.cc:1403] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.000s
I20251028 03:00:00.455785 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b1c390b850f444aca6a3856952225164" } modify_external_catalogs: true
I20251028 03:00:00.456872 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b1c390b850f444aca6a3856952225164" } modify_external_catalogs: true
I20251028 03:00:00.457180 8544 raft_consensus.cc:359] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.457276 8544 raft_consensus.cc:385] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.457312 8544 raft_consensus.cc:740] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.457434 8544 consensus_queue.cc:260] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.457429 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f73f3b69b069406a899dd5934efb41a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.457706 8544 ts_tablet_manager.cc:1434] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 03:00:00.457808 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f73f3b69b069406a899dd5934efb41a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.457895 8544 tablet_bootstrap.cc:492] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.458333 8356 leader_election.cc:304] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.458498 9840 raft_consensus.cc:2804] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.458541 9840 raft_consensus.cc:493] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.458576 9840 raft_consensus.cc:3060] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.458899 8544 tablet_bootstrap.cc:654] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.459317 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 1c41d49f9eba40a48bd6900671297db9 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:00.459630 9840 raft_consensus.cc:515] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.459664 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f5fe354dcfca4c57b4102be932490256"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.460003 9840 leader_election.cc:290] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:00.460027 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f5fe354dcfca4c57b4102be932490256 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.460675 8273 raft_consensus.cc:3060] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:00.461225 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f62b981b9b1a44bb96bbc90f924c3b64 consensus state has no leader
I20251028 03:00:00.461226 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f28ca2cf41564792b49940aebc9b2c98" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:00.461333 8271 raft_consensus.cc:3060] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.461720 8273 raft_consensus.cc:1275] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:00.462129 9839 consensus_queue.cc:1048] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.462252 8271 raft_consensus.cc:2468] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.462683 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6c1263fbbde44afa8012fae4052d8795"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.463033 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c1263fbbde44afa8012fae4052d8795 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.463213 8356 leader_election.cc:304] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.463402 9839 raft_consensus.cc:2804] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.463476 9839 raft_consensus.cc:697] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:00.463595 9839 consensus_queue.cc:237] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.463860 8143 raft_consensus.cc:1275] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:00.464088 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f28ca2cf41564792b49940aebc9b2c98" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.464191 8145 raft_consensus.cc:3060] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.464327 8552 tablet_replica.cc:333] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.464479 8552 raft_consensus.cc:2243] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.464582 8552 raft_consensus.cc:2272] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:00.464592 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet efbbfee9d8c44e5489e7a24fdd46f290 consensus state has no leader
I20251028 03:00:00.465247 8552 ts_tablet_manager.cc:1916] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.465329 8145 raft_consensus.cc:2468] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.465404 8546 tablet_bootstrap.cc:654] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.466463 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f28ca2cf41564792b49940aebc9b2c98" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.466578 8147 raft_consensus.cc:2376] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1. Re-sending same reply.
I20251028 03:00:00.467080 9958 raft_consensus.cc:493] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.467175 9958 raft_consensus.cc:515] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.467494 8552 ts_tablet_manager.cc:1929] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.467557 8552 log.cc:1199] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/302e0ed455854eb2b661afc2f2b6921d
I20251028 03:00:00.467583 9958 leader_election.cc:290] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:00.468597 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1979906599754f7ba049c7a9b23bf26d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.469144 8544 tablet_bootstrap.cc:492] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.469178 9839 consensus_queue.cc:1048] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.469229 8544 ts_tablet_manager.cc:1403] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 03:00:00.469537 8552 ts_tablet_manager.cc:1950] T 302e0ed455854eb2b661afc2f2b6921d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.469731 8544 raft_consensus.cc:359] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.469836 8544 raft_consensus.cc:385] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.469867 8544 raft_consensus.cc:740] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.470001 8544 consensus_queue.cc:260] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.470132 8552 tablet_replica.cc:333] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.470273 8552 raft_consensus.cc:2243] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.470307 8544 ts_tablet_manager.cc:1434] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.470366 8552 raft_consensus.cc:2272] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.470506 8544 tablet_bootstrap.cc:492] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.471063 8552 ts_tablet_manager.cc:1916] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.472958 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1979906599754f7ba049c7a9b23bf26d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.473089 8147 raft_consensus.cc:2393] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
W20251028 03:00:00.473258 9912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251028 03:00:00.473320 9912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251028 03:00:00.473351 9912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251028 03:00:00.473377 9912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251028 03:00:00.473834 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 2a2ebe28dbd2439e963319ec6e515635 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.474922 8218 leader_election.cc:304] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:00.475143 9884 raft_consensus.cc:3060] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.476547 9884 raft_consensus.cc:2749] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.476825 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 302e0ed455854eb2b661afc2f2b6921d (table default.Table_7e5a0a5655db40d8957f85af13a091b5 [id=25691f2b0c2a483c96fe4eaeebc338f9]) successfully deleted
I20251028 03:00:00.477481 8552 ts_tablet_manager.cc:1929] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.477543 8552 log.cc:1199] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/55e72a7f8aa842049da7aeaaa7001a28
I20251028 03:00:00.477556 8544 tablet_bootstrap.cc:654] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.477670 8550 ts_tablet_manager.cc:1916] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.477751 8552 ts_tablet_manager.cc:1950] T 55e72a7f8aa842049da7aeaaa7001a28 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.478523 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 55e72a7f8aa842049da7aeaaa7001a28 (table default.Table_ea6294c26df241f59b9adcc8f2a9eddc [id=881361e7a7744eeea8e430acff290a31]) successfully deleted
I20251028 03:00:00.478951 8552 tablet_replica.cc:333] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.479074 8552 raft_consensus.cc:2243] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.479162 8552 raft_consensus.cc:2272] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.479647 8545 tablet_bootstrap.cc:492] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.479735 8545 ts_tablet_manager.cc:1403] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.032s user 0.000s sys 0.002s
I20251028 03:00:00.480283 8545 raft_consensus.cc:359] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.480387 8545 raft_consensus.cc:385] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.480427 8545 raft_consensus.cc:740] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.480557 8545 consensus_queue.cc:260] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.480870 8545 ts_tablet_manager.cc:1434] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.481102 8545 tablet_bootstrap.cc:492] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.481475 8552 ts_tablet_manager.cc:1916] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.482285 8545 tablet_bootstrap.cc:654] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.483217 7962 catalog_manager.cc:5649] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } }
I20251028 03:00:00.483608 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 00550a40f04046d19e9112814372da6c (table default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]). Expected version 1 got 0
I20251028 03:00:00.484386 8552 ts_tablet_manager.cc:1929] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.484470 8552 log.cc:1199] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/d39008ac6324451a9d1ed54a2ccc3abf
I20251028 03:00:00.484684 8552 ts_tablet_manager.cc:1950] T d39008ac6324451a9d1ed54a2ccc3abf P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.485386 8546 tablet_bootstrap.cc:492] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.485496 8546 ts_tablet_manager.cc:1403] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.001s
I20251028 03:00:00.485791 8544 tablet_bootstrap.cc:492] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.485886 8544 ts_tablet_manager.cc:1403] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 03:00:00.485999 8546 raft_consensus.cc:359] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.486095 8546 raft_consensus.cc:385] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.486127 8546 raft_consensus.cc:740] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.486243 8546 consensus_queue.cc:260] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.486387 8544 raft_consensus.cc:359] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.486439 9840 raft_consensus.cc:493] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.486490 8544 raft_consensus.cc:385] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.486521 8544 raft_consensus.cc:740] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.486518 9840 raft_consensus.cc:515] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.486652 8544 consensus_queue.cc:260] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.486799 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 0d42a5987fa540ce86afdbbe1922896e
I20251028 03:00:00.486914 9840 leader_election.cc:290] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.486969 8544 ts_tablet_manager.cc:1434] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.487185 8544 tablet_bootstrap.cc:492] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.487924 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet d39008ac6324451a9d1ed54a2ccc3abf (table default.Table_f3795491c9e547af98943428398101b2 [id=8fe996a5bf644b1a9a0d2d3894445a56]) successfully deleted
W20251028 03:00:00.488940 8354 leader_election.cc:343] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: d39008ac6324451a9d1ed54a2ccc3abf
I20251028 03:00:00.489851 9906 tablet.cc:1722] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:00.490768 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d42a5987fa540ce86afdbbe1922896e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:00.490962 8356 leader_election.cc:343] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: d39008ac6324451a9d1ed54a2ccc3abf
I20251028 03:00:00.491031 8356 leader_election.cc:304] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:00.491127 8544 tablet_bootstrap.cc:654] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.491245 9639 raft_consensus.cc:2749] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.491395 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d42a5987fa540ce86afdbbe1922896e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.492174 9909 tablet.cc:1722] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:00.493767 8546 ts_tablet_manager.cc:1434] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
W20251028 03:00:00.494441 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 0d42a5987fa540ce86afdbbe1922896e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0d42a5987fa540ce86afdbbe1922896e already in progress: creating tablet
W20251028 03:00:00.494690 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 0d42a5987fa540ce86afdbbe1922896e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0d42a5987fa540ce86afdbbe1922896e already in progress: creating tablet
I20251028 03:00:00.494786 8546 tablet_bootstrap.cc:492] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.495541 9639 raft_consensus.cc:493] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.495620 9639 raft_consensus.cc:515] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.495963 9639 leader_election.cc:290] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.496426 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d3e422c7ae64b25960714ecc926046a" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.496436 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 0d42a5987fa540ce86afdbbe1922896e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.496557 8145 raft_consensus.cc:2468] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:00.498826 8354 leader_election.cc:304] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:00.498988 9639 raft_consensus.cc:2804] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.499029 9639 raft_consensus.cc:493] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.499064 9639 raft_consensus.cc:3060] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.500121 9639 raft_consensus.cc:515] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.500458 9639 leader_election.cc:290] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.501231 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d3e422c7ae64b25960714ecc926046a" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.501348 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d3e422c7ae64b25960714ecc926046a" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.501333 9912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.154.148:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.7.154.190:39091,127.7.154.189:45473,127.7.154.188:44237
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.154.190:39091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.154.190
--webserver_port=40487
--never_fsync=true
--heap_profile_path=/tmp/kudu.9912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 60f5e5267b92c39485a66121d3ce3cc7ef57b0e0
build type DEBUG
built by None at 28 Oct 2025 02:43:12 UTC on 24a791456cd2
build id 8750
I20251028 03:00:00.501231 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1d3e422c7ae64b25960714ecc926046a" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:00.501454 8145 raft_consensus.cc:3060] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.502424 8145 raft_consensus.cc:2468] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.502889 9912 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251028 03:00:00.503453 8544 tablet_bootstrap.cc:492] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.503433 8354 leader_election.cc:304] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:00.503551 8544 ts_tablet_manager.cc:1403] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 03:00:00.503837 9190 raft_consensus.cc:2804] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.503931 9190 raft_consensus.cc:697] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:00.503995 8544 raft_consensus.cc:359] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.504103 8544 raft_consensus.cc:385] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.504061 9190 consensus_queue.cc:237] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.504143 8544 raft_consensus.cc:740] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.504256 8544 consensus_queue.cc:260] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.504528 8544 ts_tablet_manager.cc:1434] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.504699 8544 tablet_bootstrap.cc:492] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.504711 9912 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251028 03:00:00.505396 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet b9b5fb938357499b8888ba29d23612bd (table default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]). Expected version 1 got 0
I20251028 03:00:00.505545 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 464e6b3483bc4816bfa844c52fd3ca53 (table default.Table_e95d99935d7d4d94a16f5a826847ce32 [id=71026b1300014ba6a66c3431c05cc86c]). Expected version 1 got 0
I20251028 03:00:00.505705 7960 catalog_manager.cc:5649] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:00.505779 9913 tablet.cc:1722] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:00.505877 8544 tablet_bootstrap.cc:654] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:00.506351 8356 leader_election.cc:343] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:00.506546 8356 leader_election.cc:343] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.508904 8544 tablet_bootstrap.cc:492] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.508987 8544 ts_tablet_manager.cc:1403] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 03:00:00.509454 8544 raft_consensus.cc:359] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.509550 8544 raft_consensus.cc:385] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.509578 8544 raft_consensus.cc:740] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.509692 8544 consensus_queue.cc:260] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.510040 8544 ts_tablet_manager.cc:1434] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.510160 8546 tablet_bootstrap.cc:654] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.510231 8544 tablet_bootstrap.cc:492] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.510605 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bf9795bd59b42ea997a97c84e1ea111 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.510741 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 2893b3a6e8854187b837a7e6fef58c4e with delete_type TABLET_DATA_DELETED (replaced by tablet f5e7e0614af04cc2a22130906d062a79) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.510819 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 5155389485d54c1b82c45eb65364620f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.512452 8546 tablet_bootstrap.cc:492] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.512560 8546 ts_tablet_manager.cc:1403] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.002s
I20251028 03:00:00.512552 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 8192667f14a44ec296d28efa6057a44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.513068 8546 raft_consensus.cc:359] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.513180 8546 raft_consensus.cc:385] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.513218 8546 raft_consensus.cc:740] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.513337 8546 consensus_queue.cc:260] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.513626 8546 ts_tablet_manager.cc:1434] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.513852 8546 tablet_bootstrap.cc:492] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.514313 8544 tablet_bootstrap.cc:654] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.514811 8546 tablet_bootstrap.cc:654] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.516000 8546 tablet_bootstrap.cc:492] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.516077 8546 ts_tablet_manager.cc:1403] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20251028 03:00:00.516528 8546 raft_consensus.cc:359] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.516644 8546 raft_consensus.cc:385] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.516688 8546 raft_consensus.cc:740] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.516831 8546 consensus_queue.cc:260] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.517113 8546 ts_tablet_manager.cc:1434] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:00.517316 8546 tablet_bootstrap.cc:492] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.517486 9788 raft_consensus.cc:493] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.517570 9788 raft_consensus.cc:515] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.517979 9788 leader_election.cc:290] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.518422 8546 tablet_bootstrap.cc:654] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.518378 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0fcabc4a2fd44043be5451046ff5bd97" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:00.519086 8083 leader_election.cc:343] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.519673 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0fcabc4a2fd44043be5451046ff5bd97" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.520006 8083 leader_election.cc:343] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.520080 8083 leader_election.cc:304] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.520328 9788 raft_consensus.cc:2749] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.521526 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet b9b5fb938357499b8888ba29d23612bd (table default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]). Expected version 1 got 0
I20251028 03:00:00.521641 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 69e0358b593f4b55b6b4ec859e2c021d (table default.Table_49f2ad3b1c934eaeab556927604ddf1b [id=e88ffc49924340aa92a3aedd631330bf]). Expected version 1 got 0
I20251028 03:00:00.522600 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet c120a3d56f6b41c28c7f7dcd84359a60 with delete_type TABLET_DATA_DELETED (replaced by tablet dea4de43bc384ac8bd55472aa1e59c7e) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.522763 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet d39008ac6324451a9d1ed54a2ccc3abf with delete_type TABLET_DATA_DELETED (replaced by tablet 985a0daa0a544216bbb06b1bc9d19231) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.522836 8551 tablet_replica.cc:333] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.522735 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 with delete_type TABLET_DATA_DELETED (replaced by tablet d37c267aaee7412b869753164556b9c0) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.522981 8551 raft_consensus.cc:2243] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.523088 8551 raft_consensus.cc:2272] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.523787 8551 ts_tablet_manager.cc:1916] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.524966 8546 tablet_bootstrap.cc:492] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.525089 8546 ts_tablet_manager.cc:1403] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20251028 03:00:00.524966 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 0e19a2bdc7024b0c9fe84f8983f9d292 with delete_type TABLET_DATA_DELETED (replaced by tablet d37c267aaee7412b869753164556b9c0) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.525314 8552 tablet_replica.cc:333] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.525431 8552 raft_consensus.cc:2243] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.525547 8552 raft_consensus.cc:2272] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.526014 8552 ts_tablet_manager.cc:1916] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.526293 8551 ts_tablet_manager.cc:1929] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.526391 8551 log.cc:1199] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c120a3d56f6b41c28c7f7dcd84359a60
I20251028 03:00:00.526427 7965 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 464e6b3483bc4816bfa844c52fd3ca53 (table default.Table_e95d99935d7d4d94a16f5a826847ce32 [id=71026b1300014ba6a66c3431c05cc86c]). Expected version 1 got 0
I20251028 03:00:00.526612 8551 ts_tablet_manager.cc:1950] T c120a3d56f6b41c28c7f7dcd84359a60 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.527249 8551 tablet_replica.cc:333] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.527410 8551 raft_consensus.cc:2243] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.527415 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet c120a3d56f6b41c28c7f7dcd84359a60 (table default.Table_13dbc4fadcfd41c8945172798836fb62 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]) successfully deleted
I20251028 03:00:00.527513 8551 raft_consensus.cc:2272] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.528010 8551 ts_tablet_manager.cc:1916] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.528023 8552 ts_tablet_manager.cc:1929] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.528100 8552 log.cc:1199] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/0e19a2bdc7024b0c9fe84f8983f9d292
I20251028 03:00:00.528307 8552 ts_tablet_manager.cc:1950] T 0e19a2bdc7024b0c9fe84f8983f9d292 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.528906 9884 raft_consensus.cc:493] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.528935 8546 raft_consensus.cc:359] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.529014 9884 raft_consensus.cc:515] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.529398 9884 leader_election.cc:290] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.529810 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b9b5fb938357499b8888ba29d23612bd" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.529896 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b9b5fb938357499b8888ba29d23612bd" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.529974 8419 raft_consensus.cc:2468] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:00.530043 8145 raft_consensus.cc:2468] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:00.530270 9788 raft_consensus.cc:493] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.530318 8220 leader_election.cc:304] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.530342 9788 raft_consensus.cc:515] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.530551 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 0e19a2bdc7024b0c9fe84f8983f9d292 (table default.Table_4f7a6b08bb2b4cee90e54afd8f1b801a [id=243b57c791df41819156363e8607623d]) successfully deleted
I20251028 03:00:00.530700 9788 leader_election.cc:290] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.530757 9884 raft_consensus.cc:2804] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.530823 9884 raft_consensus.cc:493] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20251028 03:00:00.530790 10024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 03:00:00.530869 9884 raft_consensus.cc:3060] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:00.531083 10026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251028 03:00:00.531378 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00550a40f04046d19e9112814372da6c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.531486 10023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251028 03:00:00.531827 8083 leader_election.cc:343] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.531971 7966 catalog_manager.cc:5649] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 03:00:00.532217 7966 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 1d3e422c7ae64b25960714ecc926046a (table default.Table_d9b8ba18e24848b1b8511a66b8c7cf2f [id=181b50c2d29a4b17b8189e3abb10a420]). Expected version 2 got 0
I20251028 03:00:00.532445 7966 catalog_manager.cc:5649] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:00.532524 9912 server_base.cc:1047] running on GCE node
I20251028 03:00:00.533056 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00550a40f04046d19e9112814372da6c" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.532610 9884 raft_consensus.cc:515] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.533332 9912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20251028 03:00:00.533453 8083 leader_election.cc:343] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.533520 8083 leader_election.cc:304] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.533610 9884 leader_election.cc:290] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.533735 9788 raft_consensus.cc:2749] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.533949 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b9b5fb938357499b8888ba29d23612bd" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.533977 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b9b5fb938357499b8888ba29d23612bd" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:00.534070 8145 raft_consensus.cc:3060] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.534076 8419 raft_consensus.cc:3060] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.535063 8419 raft_consensus.cc:2468] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:00.535115 8145 raft_consensus.cc:2468] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:00.535580 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 225ef744f6724f158981e051f2efbb43 (DEFAULT_TABLE table=default.Table_a2f9a31f1859418e9dca1b761f5025e9 [id=08ff04ccebfe4cefbc475d0bb686a22f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.535934 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225ef744f6724f158981e051f2efbb43. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.536690 8218 leader_election.cc:304] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:00.537078 9884 raft_consensus.cc:2804] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.537407 9884 raft_consensus.cc:697] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:00.537555 9884 consensus_queue.cc:237] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.538574 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 225ef744f6724f158981e051f2efbb43 (DEFAULT_TABLE table=default.Table_a2f9a31f1859418e9dca1b761f5025e9 [id=08ff04ccebfe4cefbc475d0bb686a22f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.538837 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225ef744f6724f158981e051f2efbb43. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.538846 9912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251028 03:00:00.539271 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 225ef744f6724f158981e051f2efbb43 (DEFAULT_TABLE table=default.Table_a2f9a31f1859418e9dca1b761f5025e9 [id=08ff04ccebfe4cefbc475d0bb686a22f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.539546 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225ef744f6724f158981e051f2efbb43. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.529040 8546 raft_consensus.cc:385] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.542235 8546 raft_consensus.cc:740] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.542394 8546 consensus_queue.cc:260] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.544025 8550 ts_tablet_manager.cc:1929] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:00.545452 8546 ts_tablet_manager.cc:1434] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.020s user 0.001s sys 0.000s
I20251028 03:00:00.544951 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_c7534bd73076465c96d28ee8be7d5800 [id=262048a24b0f47b0b3c0dee427389e4c] alter complete (version 1)
I20251028 03:00:00.548301 8546 tablet_bootstrap.cc:492] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.549484 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet b9803f4cfb9b4ef9a1954c83985d6258 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b6f62b942a94af783c026a043cc60a3) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.550041 8546 tablet_bootstrap.cc:654] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.550717 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 12eecf12cac641eaad5a9b5af5890941 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.550834 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet c120a3d56f6b41c28c7f7dcd84359a60 with delete_type TABLET_DATA_DELETED (replaced by tablet dea4de43bc384ac8bd55472aa1e59c7e) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.550999 8552 tablet_replica.cc:333] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.551162 8552 raft_consensus.cc:2243] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.551285 8552 raft_consensus.cc:2272] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.551746 8550 log.cc:1199] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/f57b4fa3ceec4557bc6d6a2125d80790
I20251028 03:00:00.552286 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c8b4b0aa30194ba495adca9138e2454e
I20251028 03:00:00.553498 8551 ts_tablet_manager.cc:1929] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.553612 8551 log.cc:1199] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/d39008ac6324451a9d1ed54a2ccc3abf
I20251028 03:00:00.553879 8551 ts_tablet_manager.cc:1950] T d39008ac6324451a9d1ed54a2ccc3abf P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.554602 8545 tablet_bootstrap.cc:492] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.554652 8551 tablet_replica.cc:333] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.554703 8545 ts_tablet_manager.cc:1403] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.074s user 0.001s sys 0.000s
I20251028 03:00:00.554852 8551 raft_consensus.cc:2243] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.554993 8551 raft_consensus.cc:2272] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.555212 8545 raft_consensus.cc:359] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.555370 8545 raft_consensus.cc:385] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.555419 8545 raft_consensus.cc:740] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.555491 8551 ts_tablet_manager.cc:1916] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.555526 8545 consensus_queue.cc:260] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.555824 8545 ts_tablet_manager.cc:1434] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.556043 8545 tablet_bootstrap.cc:492] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.556551 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet c8b4b0aa30194ba495adca9138e2454e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.557179 8545 tablet_bootstrap.cc:654] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.557233 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet d39008ac6324451a9d1ed54a2ccc3abf (table default.Table_f3795491c9e547af98943428398101b2 [id=8fe996a5bf644b1a9a0d2d3894445a56]) successfully deleted
I20251028 03:00:00.557456 9912 hybrid_clock.cc:648] HybridClock initialized: now 1761620400557372 us; error 104 us; skew 500 ppm
I20251028 03:00:00.557868 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet c8b4b0aa30194ba495adca9138e2454e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.558269 8550 ts_tablet_manager.cc:1950] T f57b4fa3ceec4557bc6d6a2125d80790 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.548736 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 0fcabc4a2fd44043be5451046ff5bd97 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.558674 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet f57b4fa3ceec4557bc6d6a2125d80790 (table default.Table_1023c48e40024553b0795a912f2fdd18 [id=804788948646450483232f6817d1efaf]) successfully deleted
I20251028 03:00:00.558820 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 12eecf12cac641eaad5a9b5af5890941 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.560076 9884 consensus_queue.cc:1048] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.560359 9912 webserver.cc:492] Webserver started at http://127.7.154.190:40487/ using document root <none> and password file <none>
I20251028 03:00:00.561193 9912 fs_manager.cc:362] Metadata directory not provided
I20251028 03:00:00.561303 9912 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251028 03:00:00.563876 8545 tablet_bootstrap.cc:492] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.564002 8545 ts_tablet_manager.cc:1403] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 03:00:00.547921 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 07d3e6c640434652a010942402c54716 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e0e038d33bf45f693da251cd560ae2a) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.555660 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet c8b4b0aa30194ba495adca9138e2454e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.564508 8545 raft_consensus.cc:359] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.564618 8545 raft_consensus.cc:385] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.564648 8545 raft_consensus.cc:740] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.564761 8545 consensus_queue.cc:260] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.564893 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 8192667f14a44ec296d28efa6057a44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.565078 8545 ts_tablet_manager.cc:1434] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.565274 8545 tablet_bootstrap.cc:492] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.566839 8545 tablet_bootstrap.cc:654] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.568071 8552 ts_tablet_manager.cc:1916] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.568312 9840 consensus_queue.cc:1048] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.568801 8550 tablet_replica.cc:333] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.568958 8550 raft_consensus.cc:2243] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.569074 8550 raft_consensus.cc:2272] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:00.569789 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 05b6a671e514463bace6c3232700a110 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05b6a671e514463bace6c3232700a110 already in progress: creating tablet
I20251028 03:00:00.570868 8552 ts_tablet_manager.cc:1929] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.570945 8552 log.cc:1199] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c120a3d56f6b41c28c7f7dcd84359a60
I20251028 03:00:00.571161 8552 ts_tablet_manager.cc:1950] T c120a3d56f6b41c28c7f7dcd84359a60 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.569917 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet 5d5c68c16c3e4d37be7f9a505ab44549 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:00.572037 8552 tablet_replica.cc:333] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.572182 8552 raft_consensus.cc:2243] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.572292 8552 raft_consensus.cc:2272] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.572523 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c120a3d56f6b41c28c7f7dcd84359a60 (table default.Table_13dbc4fadcfd41c8945172798836fb62 [id=51d89e8ec9ba432e86b9a82cbfa51ec9]) successfully deleted
W20251028 03:00:00.572697 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet c8b4b0aa30194ba495adca9138e2454e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c8b4b0aa30194ba495adca9138e2454e already in progress: creating tablet
I20251028 03:00:00.572824 8552 ts_tablet_manager.cc:1916] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.573532 8550 ts_tablet_manager.cc:1916] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.575035 8552 ts_tablet_manager.cc:1929] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.575136 8552 log.cc:1199] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/12eecf12cac641eaad5a9b5af5890941
I20251028 03:00:00.575361 9912 fs_manager.cc:714] Time spent opening directory manager: real 0.012s user 0.004s sys 0.001s
I20251028 03:00:00.575389 8552 ts_tablet_manager.cc:1950] T 12eecf12cac641eaad5a9b5af5890941 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.576026 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 12eecf12cac641eaad5a9b5af5890941 (table default.Table_7ceb18d43d6c41d2b7809c8c97a5f2e7 [id=e47909e4e17049549b87f01157c35a4f]) successfully deleted
I20251028 03:00:00.578217 10038 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251028 03:00:00.578261 8551 ts_tablet_manager.cc:1929] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.578351 8551 log.cc:1199] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/0e19a2bdc7024b0c9fe84f8983f9d292
I20251028 03:00:00.578572 8551 ts_tablet_manager.cc:1950] T 0e19a2bdc7024b0c9fe84f8983f9d292 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.579576 9912 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20251028 03:00:00.579721 8551 tablet_replica.cc:333] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.579845 8545 tablet_bootstrap.cc:492] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.579865 8551 raft_consensus.cc:2243] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.579928 8545 ts_tablet_manager.cc:1403] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20251028 03:00:00.579958 8551 raft_consensus.cc:2272] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.579691 9912 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data,/tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal
uuid: "ac9bf5a198d042698c9f030a59228865"
format_stamp: "Formatted at 2025-10-28 02:59:48 on dist-test-slave-97kg"
I20251028 03:00:00.580224 9912 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251028 03:00:00.580420 8545 raft_consensus.cc:359] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.580520 8545 raft_consensus.cc:385] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.580550 8545 raft_consensus.cc:740] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.580675 8545 consensus_queue.cc:260] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.580958 8545 ts_tablet_manager.cc:1434] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:00.581174 8545 tablet_bootstrap.cc:492] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.582368 8545 tablet_bootstrap.cc:654] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.583549 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 0e19a2bdc7024b0c9fe84f8983f9d292 (table default.Table_4f7a6b08bb2b4cee90e54afd8f1b801a [id=243b57c791df41819156363e8607623d]) successfully deleted
I20251028 03:00:00.585073 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet ebf449a6d241486a892eba79568c3409 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:00.587075 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 13fb883843d141f09aa90e66f9b7bf1f (table default.Table_2547f4c4cdbd4d7f9ec732da77dc545a [id=c9c59047d50a4419a9f16431bce05cd1]). Expected version 1 got 0
I20251028 03:00:00.587370 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 1979906599754f7ba049c7a9b23bf26d (table default.Table_cdab567cf8da46e8a4d7ba2a80e1e2d9 [id=9fcc70dd88724a28b404e9d4109015aa]). Expected version 1 got 0
I20251028 03:00:00.587838 8551 ts_tablet_manager.cc:1916] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.590251 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ebf449a6d241486a892eba79568c3409
I20251028 03:00:00.590583 7966 catalog_manager.cc:6719] ReplaceTablet: tablet ebf449a6d241486a892eba79568c3409 deleted and replaced by tablet 44323698800f486990acde8ee2909260
I20251028 03:00:00.590612 8551 ts_tablet_manager.cc:1929] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.590689 8551 log.cc:1199] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/b9803f4cfb9b4ef9a1954c83985d6258
I20251028 03:00:00.590785 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet ebf449a6d241486a892eba79568c3409 with delete_type TABLET_DATA_DELETED (replaced by tablet 44323698800f486990acde8ee2909260) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.591104 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet ebf449a6d241486a892eba79568c3409 with delete_type TABLET_DATA_DELETED (replaced by tablet 44323698800f486990acde8ee2909260) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.591387 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet ebf449a6d241486a892eba79568c3409 with delete_type TABLET_DATA_DELETED (replaced by tablet 44323698800f486990acde8ee2909260) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.591503 8551 ts_tablet_manager.cc:1950] T b9803f4cfb9b4ef9a1954c83985d6258 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.591636 8544 tablet_bootstrap.cc:492] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.591727 8544 ts_tablet_manager.cc:1403] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.082s user 0.000s sys 0.002s
W20251028 03:00:00.592020 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ebf449a6d241486a892eba79568c3409 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebf449a6d241486a892eba79568c3409 already in progress: creating tablet
I20251028 03:00:00.592365 8551 tablet_replica.cc:333] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.592490 8551 raft_consensus.cc:2243] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.592569 8551 raft_consensus.cc:2272] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.592531 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet b9803f4cfb9b4ef9a1954c83985d6258 (table default.Table_849a72ead23447078b8088823de4288e [id=82ae3d70b0b544d89bd8e0ff1717f396]) successfully deleted
I20251028 03:00:00.593021 8551 ts_tablet_manager.cc:1916] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.592967 8544 raft_consensus.cc:359] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.593111 8544 raft_consensus.cc:385] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.593159 8544 raft_consensus.cc:740] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.593292 8544 consensus_queue.cc:260] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.593573 8544 ts_tablet_manager.cc:1434] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251028 03:00:00.593739 8544 tablet_bootstrap.cc:492] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.594740 8544 tablet_bootstrap.cc:654] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.595906 8551 ts_tablet_manager.cc:1929] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.595976 8551 log.cc:1199] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/12eecf12cac641eaad5a9b5af5890941
I20251028 03:00:00.596169 8551 ts_tablet_manager.cc:1950] T 12eecf12cac641eaad5a9b5af5890941 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.597100 8545 tablet_bootstrap.cc:492] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.597194 8545 ts_tablet_manager.cc:1403] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 03:00:00.597720 8545 raft_consensus.cc:359] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.597841 8545 raft_consensus.cc:385] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.597882 8545 raft_consensus.cc:740] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.598030 8545 consensus_queue.cc:260] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.598299 8545 ts_tablet_manager.cc:1434] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:00.598294 8550 ts_tablet_manager.cc:1929] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:00.598416 8550 log.cc:1199] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2a2ebe28dbd2439e963319ec6e515635
I20251028 03:00:00.598484 8545 tablet_bootstrap.cc:492] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.598850 8550 ts_tablet_manager.cc:1950] T 2a2ebe28dbd2439e963319ec6e515635 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.599347 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2a2ebe28dbd2439e963319ec6e515635 (table default.Table_b0ef051366704663b4e079ccdda63a43 [id=ec7b656f185d4b5685d6f634797b1107]) successfully deleted
I20251028 03:00:00.599545 8545 tablet_bootstrap.cc:654] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.600627 9595 consensus_queue.cc:1048] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.600735 8551 tablet_replica.cc:333] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.600857 8551 raft_consensus.cc:2243] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.600952 8551 raft_consensus.cc:2272] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.604920 8551 ts_tablet_manager.cc:1916] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.607426 9639 consensus_queue.cc:1048] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.607859 8545 tablet_bootstrap.cc:492] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.607942 8545 ts_tablet_manager.cc:1403] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 03:00:00.608024 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 12eecf12cac641eaad5a9b5af5890941 (table default.Table_7ceb18d43d6c41d2b7809c8c97a5f2e7 [id=e47909e4e17049549b87f01157c35a4f]) successfully deleted
W20251028 03:00:00.608196 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet c8b4b0aa30194ba495adca9138e2454e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c8b4b0aa30194ba495adca9138e2454e already in progress: creating tablet
I20251028 03:00:00.608379 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fe075af668b044a98e7c7128d52890a0" } modify_external_catalogs: true
I20251028 03:00:00.608415 8545 raft_consensus.cc:359] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.608474 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fe075af668b044a98e7c7128d52890a0" } modify_external_catalogs: true
I20251028 03:00:00.608507 8545 raft_consensus.cc:385] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.608537 8545 raft_consensus.cc:740] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.608649 8545 consensus_queue.cc:260] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.608894 8545 ts_tablet_manager.cc:1434] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.609071 8545 tablet_bootstrap.cc:492] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.609494 9912 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251028 03:00:00.610141 8545 tablet_bootstrap.cc:654] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.611889 8546 tablet_bootstrap.cc:492] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.611974 8546 ts_tablet_manager.cc:1403] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.064s user 0.001s sys 0.001s
I20251028 03:00:00.612424 8546 raft_consensus.cc:359] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.612527 8546 raft_consensus.cc:385] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.612556 8546 raft_consensus.cc:740] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.612661 8546 consensus_queue.cc:260] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.612957 8546 ts_tablet_manager.cc:1434] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.613149 8546 tablet_bootstrap.cc:492] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.614128 8546 tablet_bootstrap.cc:654] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.614777 8551 ts_tablet_manager.cc:1929] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.614853 8551 log.cc:1199] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8192667f14a44ec296d28efa6057a44d
I20251028 03:00:00.615090 8551 ts_tablet_manager.cc:1950] T 8192667f14a44ec296d28efa6057a44d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.616967 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8192667f14a44ec296d28efa6057a44d (table default.Table_ef415299e6ef4aa583629acaba69daf0 [id=448356df3cfe4bdcb549b652a04ff06a]) successfully deleted
I20251028 03:00:00.617548 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a5594c4b37154c58a5734fadcaf00972"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.617688 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b0aec51ddd0a4f348a95bb73c0e6d226"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.618094 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_891be01aa2b540a7b16b407884b6de92" } modify_external_catalogs: true
I20251028 03:00:00.618201 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_891be01aa2b540a7b16b407884b6de92" } modify_external_catalogs: true
I20251028 03:00:00.618512 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_62b833a367564e2db98147279f26082d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.618656 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ebf449a6d241486a892eba79568c3409 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebf449a6d241486a892eba79568c3409 already in progress: creating tablet
I20251028 03:00:00.618942 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c8fb6d535ce34fbe8032195cf9e43f57" } new_table_name: "default.Table_77a37bbdfa4f41458e6b50fb213850bc" modify_external_catalogs: true
I20251028 03:00:00.618090 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 391608f115e845e88d07ae08ea543e8e from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:00.621347 9912 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
W20251028 03:00:00.622870 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 338cf9d02142436f97ccd44313cc8199 consensus state has no leader
I20251028 03:00:00.623103 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d6d4af0e22af416ca63e8c2fda795eef" } new_table_name: "default.Table_e9f2bf5ed2684a68812f7d2c64ef9c9d" modify_external_catalogs: true
I20251028 03:00:00.623723 8544 tablet_bootstrap.cc:492] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.623831 8544 ts_tablet_manager.cc:1403] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.002s
I20251028 03:00:00.624364 8544 raft_consensus.cc:359] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.624480 8544 raft_consensus.cc:385] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.624517 8544 raft_consensus.cc:740] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.624655 8544 consensus_queue.cc:260] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.626443 8544 ts_tablet_manager.cc:1434] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 03:00:00.626654 8544 tablet_bootstrap.cc:492] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.627938 8544 tablet_bootstrap.cc:654] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.629662 9788 raft_consensus.cc:493] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.629772 9788 raft_consensus.cc:515] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.630137 9788 leader_election.cc:290] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.630839 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8192667f14a44ec296d28efa6057a44d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.630903 8083 leader_election.cc:343] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 8192667f14a44ec296d28efa6057a44d
W20251028 03:00:00.631556 8083 leader_election.cc:343] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.631618 8083 leader_election.cc:304] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.632678 9788 raft_consensus.cc:2749] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.633103 9912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251028 03:00:00.633596 8550 tablet_replica.cc:333] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.633837 8550 raft_consensus.cc:2243] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.633963 8550 raft_consensus.cc:2272] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.639145 8545 tablet_bootstrap.cc:492] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.639259 8545 ts_tablet_manager.cc:1403] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20251028 03:00:00.639695 8550 ts_tablet_manager.cc:1916] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.639833 8545 raft_consensus.cc:359] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.639971 8545 raft_consensus.cc:385] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.640022 8545 raft_consensus.cc:740] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.640149 8545 consensus_queue.cc:260] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.640437 8545 ts_tablet_manager.cc:1434] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:00.640630 8545 tablet_bootstrap.cc:492] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.643563 8545 tablet_bootstrap.cc:654] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.643976 8550 ts_tablet_manager.cc:1929] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.644057 8550 log.cc:1199] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/5bf9795bd59b42ea997a97c84e1ea111
I20251028 03:00:00.644263 8550 ts_tablet_manager.cc:1950] T 5bf9795bd59b42ea997a97c84e1ea111 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.644889 8550 tablet_replica.cc:333] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.645048 8550 raft_consensus.cc:2243] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.645149 8550 raft_consensus.cc:2272] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.646422 8546 tablet_bootstrap.cc:492] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.646531 8546 ts_tablet_manager.cc:1403] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.000s
I20251028 03:00:00.647027 8546 raft_consensus.cc:359] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.647152 8546 raft_consensus.cc:385] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.647185 8546 raft_consensus.cc:740] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.647331 8546 consensus_queue.cc:260] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.647635 8546 ts_tablet_manager.cc:1434] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.647836 8546 tablet_bootstrap.cc:492] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.649747 8546 tablet_bootstrap.cc:654] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.651471 8544 tablet_bootstrap.cc:492] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.651636 8544 ts_tablet_manager.cc:1403] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.001s
I20251028 03:00:00.652251 8544 raft_consensus.cc:359] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.652509 8544 raft_consensus.cc:385] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.652601 8544 raft_consensus.cc:740] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.652814 8544 consensus_queue.cc:260] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.656271 8544 ts_tablet_manager.cc:1434] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20251028 03:00:00.657595 9912 rpc_server.cc:307] RPC server started. Bound to: 127.7.154.190:39091
I20251028 03:00:00.659165 9912 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/master-0/data/info.pb
I20251028 03:00:00.659325 8544 tablet_bootstrap.cc:492] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.660619 8544 tablet_bootstrap.cc:654] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.660967 10100 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.154.190:39091 every 8 connection(s)
W20251028 03:00:00.661830 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_62b833a367564e2db98147279f26082d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:00.662951 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0aec51ddd0a4f348a95bb73c0e6d226 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.663968 8546 tablet_bootstrap.cc:492] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
W20251028 03:00:00.634455 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 0d42a5987fa540ce86afdbbe1922896e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0d42a5987fa540ce86afdbbe1922896e already in progress: creating tablet
I20251028 03:00:00.664095 8546 ts_tablet_manager.cc:1403] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 03:00:00.664314 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 5bf9795bd59b42ea997a97c84e1ea111 (table default.Table_7bfb43f76f8b46b292deee50a8f58280 [id=473a7ccabea349c087076b5d741690f0]) successfully deleted
I20251028 03:00:00.664650 8546 raft_consensus.cc:359] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.664764 8546 raft_consensus.cc:385] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.664801 8546 raft_consensus.cc:740] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.664937 8546 consensus_queue.cc:260] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.665473 7786 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu as pid 9912
W20251028 03:00:00.665884 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1a3132020baa4d9a83a76df511fac5c6 consensus state has no leader
I20251028 03:00:00.666065 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet 698cca4847e94a79933c644f6b1c1c8d from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:00.667075 10101 sys_catalog.cc:263] Verifying existing consensus state
I20251028 03:00:00.667677 8546 ts_tablet_manager.cc:1434] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 03:00:00.668198 8546 tablet_bootstrap.cc:492] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.669427 10101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: Bootstrap starting.
I20251028 03:00:00.669451 8546 tablet_bootstrap.cc:654] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.674361 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8acb4514466d406a89de3678169c6180" } modify_external_catalogs: true
I20251028 03:00:00.674513 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8acb4514466d406a89de3678169c6180" } modify_external_catalogs: true
I20251028 03:00:00.674754 8550 ts_tablet_manager.cc:1916] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.677735 9788 raft_consensus.cc:493] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.677855 9788 raft_consensus.cc:515] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.678046 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a4a285cc359344a59f28244ff43d0da4" } new_table_name: "default.Table_b30c7f4678f74e759917a8de86282725" modify_external_catalogs: true
I20251028 03:00:00.678254 9788 leader_election.cc:290] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 03:00:00.678869 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5594c4b37154c58a5734fadcaf00972 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.678949 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b5e3c89222754eacb6ba4773396d62a9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.679395 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b5e3c89222754eacb6ba4773396d62a9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.679942 8083 leader_election.cc:343] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:00.680065 8083 leader_election.cc:343] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.680104 8083 leader_election.cc:304] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.680614 9788 raft_consensus.cc:2749] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.685528 8550 ts_tablet_manager.cc:1929] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.685763 8550 log.cc:1199] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2893b3a6e8854187b837a7e6fef58c4e
W20251028 03:00:00.685971 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 63ff5fe6968b47119ef0eabe691dc41c consensus state has no leader
I20251028 03:00:00.686079 8550 ts_tablet_manager.cc:1950] T 2893b3a6e8854187b837a7e6fef58c4e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.686221 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_661c1422da1f48deb0f40ffb22acad0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.686581 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_661c1422da1f48deb0f40ffb22acad0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.686818 8550 tablet_replica.cc:333] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.686967 8550 raft_consensus.cc:2243] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.687064 8550 raft_consensus.cc:2272] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.687747 8550 ts_tablet_manager.cc:1916] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.689958 8550 ts_tablet_manager.cc:1929] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.690102 8550 log.cc:1199] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/5155389485d54c1b82c45eb65364620f
I20251028 03:00:00.690315 9595 raft_consensus.cc:493] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.690410 9595 raft_consensus.cc:515] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.690564 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2893b3a6e8854187b837a7e6fef58c4e (table default.Table_85b63c813dc94719b07f0e12854b164b [id=49bac53017824831928c69211121863b]) successfully deleted
I20251028 03:00:00.690830 9595 leader_election.cc:290] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.685892 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_07adbc0d0e674d0987e90d4d9bb902c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.691190 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_07adbc0d0e674d0987e90d4d9bb902c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.691478 8273 raft_consensus.cc:3060] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.691573 8550 ts_tablet_manager.cc:1950] T 5155389485d54c1b82c45eb65364620f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.692152 8145 raft_consensus.cc:1275] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:00.692356 8550 tablet_replica.cc:333] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.692540 8550 raft_consensus.cc:2243] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.692629 8273 raft_consensus.cc:1275] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:00.692677 8550 raft_consensus.cc:2272] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.692550 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8192667f14a44ec296d28efa6057a44d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.693029 9840 consensus_queue.cc:1048] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 03:00:00.693257 8220 leader_election.cc:343] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 8192667f14a44ec296d28efa6057a44d
W20251028 03:00:00.693543 8218 leader_election.cc:343] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.693604 8218 leader_election.cc:304] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:00.693564 9840 consensus_queue.cc:1048] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.693821 8553 raft_consensus.cc:2749] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.694274 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 5155389485d54c1b82c45eb65364620f (table default.Table_b359d8a7e6724c21a2f0de83ddd841b7 [id=eb0e103661fc4bc7b27bb78fc333fd56]) successfully deleted
I20251028 03:00:00.699885 7966 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 2bf100407b034626b4a9f7e2340f9cf0 (table default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262]). Expected version 2 got 0
I20251028 03:00:00.700551 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f170e997649f4ff3a51b21a5ee733768" } new_table_name: "default.Table_aa03ff4bf2694e7eabc52863451cba52" modify_external_catalogs: true
I20251028 03:00:00.700580 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f08ece6c25eb42df91132036c01aac92"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.700938 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f08ece6c25eb42df91132036c01aac92 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.701404 7961 catalog_manager.cc:5649] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:00.701586 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet b9b5fb938357499b8888ba29d23612bd (table default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]). Expected version 1 got 0
I20251028 03:00:00.701706 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 1d3e422c7ae64b25960714ecc926046a (table default.Table_d9b8ba18e24848b1b8511a66b8c7cf2f [id=181b50c2d29a4b17b8189e3abb10a420]). Expected version 2 got 0
I20251028 03:00:00.702895 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 2e221005767b4f0e9d05b4a8baa7b928 (DEFAULT_TABLE table=default.Table_6c1263fbbde44afa8012fae4052d8795 [id=35248a5b93b0420ba418cddfb4bb267b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.703227 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e221005767b4f0e9d05b4a8baa7b928. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.703384 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 1259c359b61649d194fe46390d032c7e (DEFAULT_TABLE table=default.Table_e529e86b94f949c8a4156848587444fb [id=504019bf90394f90a823e594c55d6ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.703400 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 98a2d83563a145b7b5eec738d9d8a0e9 (DEFAULT_TABLE table=default.Table_aaeb21a72b4545c090ea639e8630a4d5 [id=85591b7515184c299095dae23fe2874a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.703632 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98a2d83563a145b7b5eec738d9d8a0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.703631 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1259c359b61649d194fe46390d032c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.705770 8553 raft_consensus.cc:493] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.705883 8390 tablet_service.cc:1505] Processing CreateTablet for tablet ce4cb3813e7045fe9eb474052d1bdf1a (DEFAULT_TABLE table=default.Table_c6a86fd3f7c140eb852bd8a9bd48f8ea [id=a55f22718f3f485480cc51c6e989c039]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.705864 8257 tablet_service.cc:1505] Processing CreateTablet for tablet ce4cb3813e7045fe9eb474052d1bdf1a (DEFAULT_TABLE table=default.Table_c6a86fd3f7c140eb852bd8a9bd48f8ea [id=a55f22718f3f485480cc51c6e989c039]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.705869 8553 raft_consensus.cc:515] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.706135 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce4cb3813e7045fe9eb474052d1bdf1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.706140 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce4cb3813e7045fe9eb474052d1bdf1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.706308 8553 leader_election.cc:290] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.708264 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 1259c359b61649d194fe46390d032c7e (DEFAULT_TABLE table=default.Table_e529e86b94f949c8a4156848587444fb [id=504019bf90394f90a823e594c55d6ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.708376 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 98a2d83563a145b7b5eec738d9d8a0e9 (DEFAULT_TABLE table=default.Table_aaeb21a72b4545c090ea639e8630a4d5 [id=85591b7515184c299095dae23fe2874a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.708489 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1259c359b61649d194fe46390d032c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.708601 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98a2d83563a145b7b5eec738d9d8a0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.711227 9888 raft_consensus.cc:493] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.711342 9888 raft_consensus.cc:515] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.711706 9888 leader_election.cc:290] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.712327 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0fbcdef1007456997db9fcf610f1e00" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.712370 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0fbcdef1007456997db9fcf610f1e00" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.712707 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 2e221005767b4f0e9d05b4a8baa7b928 (DEFAULT_TABLE table=default.Table_6c1263fbbde44afa8012fae4052d8795 [id=35248a5b93b0420ba418cddfb4bb267b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.712949 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e221005767b4f0e9d05b4a8baa7b928. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:00.715127 8083 leader_election.cc:343] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.715641 8544 tablet_bootstrap.cc:492] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.715742 8544 ts_tablet_manager.cc:1403] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.000s
I20251028 03:00:00.716307 8544 raft_consensus.cc:359] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.716421 8544 raft_consensus.cc:385] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.716456 8544 raft_consensus.cc:740] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.716583 8544 consensus_queue.cc:260] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.716902 8544 ts_tablet_manager.cc:1434] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:00.717103 8544 tablet_bootstrap.cc:492] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.715746 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "464e6b3483bc4816bfa844c52fd3ca53" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.717296 8145 raft_consensus.cc:2468] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:00.717589 8218 leader_election.cc:304] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:00.717854 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "464e6b3483bc4816bfa844c52fd3ca53" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.717964 8419 raft_consensus.cc:2468] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:00.718209 8544 tablet_bootstrap.cc:654] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:00.718556 8083 leader_election.cc:343] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.718627 8083 leader_election.cc:304] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.718830 9788 raft_consensus.cc:2749] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.719077 8553 raft_consensus.cc:2804] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.719138 8553 raft_consensus.cc:493] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.719178 8553 raft_consensus.cc:3060] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.720310 8553 raft_consensus.cc:515] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.720681 8553 leader_election.cc:290] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.722711 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "464e6b3483bc4816bfa844c52fd3ca53" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.722826 8145 raft_consensus.cc:3060] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.723486 9906 tablet.cc:1722] T 1d3e422c7ae64b25960714ecc926046a P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:00.724790 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 11f422fe1c6e437f9f95c328c9c2a000 (DEFAULT_TABLE table=default.Table_37a7da4dda72400d81691a3598906d38 [id=1d2560979b574b319b04eea3fd3348f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.725072 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f422fe1c6e437f9f95c328c9c2a000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.725045 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 5409821cb42c4c989ed7dde7ec2a8dea (DEFAULT_TABLE table=default.Table_356d26cd8d9744b290961390a0918f05 [id=3bd5ce49b0dc4c01b463630686ea69db]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.725286 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5409821cb42c4c989ed7dde7ec2a8dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.725417 8550 ts_tablet_manager.cc:1916] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.725453 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 203e4c9d73fb46b7bf09e5a86896117e (DEFAULT_TABLE table=default.Table_f73f3b69b069406a899dd5934efb41a3 [id=a14c7753deac45a1b1f83761dd0f0f17]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.725670 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 203e4c9d73fb46b7bf09e5a86896117e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.725644 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 2851b9c52e144c85a5f6673a8723c289 (DEFAULT_TABLE table=default.Table_e29ed40d642b400192ea2dc90e9c28b0 [id=c7d8d868cfb14501aa49ba406a00f5ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.725824 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2851b9c52e144c85a5f6673a8723c289. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.725983 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 86a24503546740a9b22d8b35e018124c (DEFAULT_TABLE table=default.Table_6d5acf2d672a48d3bc24b9bd2d20a174 [id=c91a69f52e534459bbdfad3462b8b063]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.726182 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86a24503546740a9b22d8b35e018124c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.727497 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 9ad86438692e44b4915462c39c2c911d (DEFAULT_TABLE table=default.Table_f5fe354dcfca4c57b4102be932490256 [id=e07192591d4549b5ae73e5fc4be6e069]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.727712 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad86438692e44b4915462c39c2c911d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.727900 8546 tablet_bootstrap.cc:492] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.727881 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "464e6b3483bc4816bfa844c52fd3ca53" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:00.727978 8546 ts_tablet_manager.cc:1403] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.060s user 0.001s sys 0.001s
I20251028 03:00:00.728008 8419 raft_consensus.cc:3060] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.728505 8546 raft_consensus.cc:359] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.728598 8546 raft_consensus.cc:385] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.728628 8546 raft_consensus.cc:740] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.728757 8546 consensus_queue.cc:260] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.729044 8419 raft_consensus.cc:2468] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:00.729943 8220 leader_election.cc:304] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.730104 9595 raft_consensus.cc:2804] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.730170 9595 raft_consensus.cc:697] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:00.730297 9595 consensus_queue.cc:237] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.730625 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 5409821cb42c4c989ed7dde7ec2a8dea (DEFAULT_TABLE table=default.Table_356d26cd8d9744b290961390a0918f05 [id=3bd5ce49b0dc4c01b463630686ea69db]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.730896 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5409821cb42c4c989ed7dde7ec2a8dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.731720 9788 raft_consensus.cc:493] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.731820 9788 raft_consensus.cc:515] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.732230 9788 leader_election.cc:290] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.733186 8389 tablet_service.cc:1505] Processing CreateTablet for tablet 2851b9c52e144c85a5f6673a8723c289 (DEFAULT_TABLE table=default.Table_e29ed40d642b400192ea2dc90e9c28b0 [id=c7d8d868cfb14501aa49ba406a00f5ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.733443 8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2851b9c52e144c85a5f6673a8723c289. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.735265 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ab2fe9132441c1834aea22561f02e9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.735515 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 9ad86438692e44b4915462c39c2c911d (DEFAULT_TABLE table=default.Table_f5fe354dcfca4c57b4102be932490256 [id=e07192591d4549b5ae73e5fc4be6e069]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.735733 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad86438692e44b4915462c39c2c911d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.735731 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 1259c359b61649d194fe46390d032c7e (DEFAULT_TABLE table=default.Table_e529e86b94f949c8a4156848587444fb [id=504019bf90394f90a823e594c55d6ca4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.736001 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1259c359b61649d194fe46390d032c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.737123 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 2e221005767b4f0e9d05b4a8baa7b928 (DEFAULT_TABLE table=default.Table_6c1263fbbde44afa8012fae4052d8795 [id=35248a5b93b0420ba418cddfb4bb267b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.737351 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e221005767b4f0e9d05b4a8baa7b928. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.738293 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 98a2d83563a145b7b5eec738d9d8a0e9 (DEFAULT_TABLE table=default.Table_aaeb21a72b4545c090ea639e8630a4d5 [id=85591b7515184c299095dae23fe2874a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.738509 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98a2d83563a145b7b5eec738d9d8a0e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.739485 8127 tablet_service.cc:1505] Processing CreateTablet for tablet ce4cb3813e7045fe9eb474052d1bdf1a (DEFAULT_TABLE table=default.Table_c6a86fd3f7c140eb852bd8a9bd48f8ea [id=a55f22718f3f485480cc51c6e989c039]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.739696 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce4cb3813e7045fe9eb474052d1bdf1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.740605 8115 tablet_service.cc:1505] Processing CreateTablet for tablet 203e4c9d73fb46b7bf09e5a86896117e (DEFAULT_TABLE table=default.Table_f73f3b69b069406a899dd5934efb41a3 [id=a14c7753deac45a1b1f83761dd0f0f17]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.740821 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 203e4c9d73fb46b7bf09e5a86896117e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.741428 9913 tablet.cc:1722] T 1d3e422c7ae64b25960714ecc926046a P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:00.741741 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 11f422fe1c6e437f9f95c328c9c2a000 (DEFAULT_TABLE table=default.Table_37a7da4dda72400d81691a3598906d38 [id=1d2560979b574b319b04eea3fd3348f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.741966 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f422fe1c6e437f9f95c328c9c2a000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.742453 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ab2fe9132441c1834aea22561f02e9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.742762 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 6758750bd7054b38bc3afabd2adaeb32 (DEFAULT_TABLE table=default.Table_9b47e45dc8f8444e8be57929ceef64d4 [id=947344a95a0d476dbb8056855ab25417]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.742980 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6758750bd7054b38bc3afabd2adaeb32. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.744108 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 5409821cb42c4c989ed7dde7ec2a8dea (DEFAULT_TABLE table=default.Table_356d26cd8d9744b290961390a0918f05 [id=3bd5ce49b0dc4c01b463630686ea69db]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.744323 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5409821cb42c4c989ed7dde7ec2a8dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.744966 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 86a24503546740a9b22d8b35e018124c (DEFAULT_TABLE table=default.Table_6d5acf2d672a48d3bc24b9bd2d20a174 [id=c91a69f52e534459bbdfad3462b8b063]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.745191 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86a24503546740a9b22d8b35e018124c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.745296 8145 raft_consensus.cc:2468] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:00.746214 9909 tablet.cc:1722] T 1d3e422c7ae64b25960714ecc926046a P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:00.746305 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 2851b9c52e144c85a5f6673a8723c289 (DEFAULT_TABLE table=default.Table_e29ed40d642b400192ea2dc90e9c28b0 [id=c7d8d868cfb14501aa49ba406a00f5ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.746490 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2851b9c52e144c85a5f6673a8723c289. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:00.747396 8083 leader_election.cc:343] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.747493 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 9ad86438692e44b4915462c39c2c911d (DEFAULT_TABLE table=default.Table_f5fe354dcfca4c57b4102be932490256 [id=e07192591d4549b5ae73e5fc4be6e069]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 03:00:00.747537 8083 leader_election.cc:343] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.747601 8083 leader_election.cc:304] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.747699 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad86438692e44b4915462c39c2c911d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.748087 9788 raft_consensus.cc:2749] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.749382 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 203e4c9d73fb46b7bf09e5a86896117e (DEFAULT_TABLE table=default.Table_f73f3b69b069406a899dd5934efb41a3 [id=a14c7753deac45a1b1f83761dd0f0f17]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.749643 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 203e4c9d73fb46b7bf09e5a86896117e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.750988 9190 consensus_queue.cc:1048] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.752429 8390 tablet_service.cc:1505] Processing CreateTablet for tablet 6758750bd7054b38bc3afabd2adaeb32 (DEFAULT_TABLE table=default.Table_9b47e45dc8f8444e8be57929ceef64d4 [id=947344a95a0d476dbb8056855ab25417]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.752651 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6758750bd7054b38bc3afabd2adaeb32. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.753605 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 86a24503546740a9b22d8b35e018124c (DEFAULT_TABLE table=default.Table_6d5acf2d672a48d3bc24b9bd2d20a174 [id=c91a69f52e534459bbdfad3462b8b063]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.753844 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86a24503546740a9b22d8b35e018124c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.755052 9788 raft_consensus.cc:493] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.755137 9788 raft_consensus.cc:515] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.755656 9788 leader_election.cc:290] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.757053 8546 ts_tablet_manager.cc:1434] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.029s user 0.001s sys 0.000s
I20251028 03:00:00.757297 8546 tablet_bootstrap.cc:492] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.758844 8546 tablet_bootstrap.cc:654] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.759822 9639 consensus_queue.cc:1048] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.763854 8546 tablet_bootstrap.cc:492] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.764025 8546 ts_tablet_manager.cc:1403] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.001s
I20251028 03:00:00.764729 9788 raft_consensus.cc:493] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.764844 9788 raft_consensus.cc:515] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.765009 8546 raft_consensus.cc:359] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.765117 8546 raft_consensus.cc:385] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.765151 8546 raft_consensus.cc:740] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.765223 9788 leader_election.cc:290] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.765301 8546 consensus_queue.cc:260] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.766067 8546 ts_tablet_manager.cc:1434] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:00.766319 8546 tablet_bootstrap.cc:492] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.766916 9595 raft_consensus.cc:493] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.766994 9595 raft_consensus.cc:515] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.767359 9595 leader_election.cc:290] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.767650 8546 tablet_bootstrap.cc:654] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.768186 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4125bb9e525c4e7d91ae0862abc44584" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.768826 8546 tablet_bootstrap.cc:492] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.768848 10101 log.cc:826] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: Log is configured to *not* fsync() on all Append() calls
I20251028 03:00:00.768913 8546 ts_tablet_manager.cc:1403] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 03:00:00.769357 8546 raft_consensus.cc:359] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.769448 8546 raft_consensus.cc:385] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.769479 8546 raft_consensus.cc:740] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.769589 8546 consensus_queue.cc:260] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.769902 8546 ts_tablet_manager.cc:1434] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.769937 8271 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "66d385494c7a4fb8a6501519925be7d4" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.770074 8546 tablet_bootstrap.cc:492] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.770023 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4125bb9e525c4e7d91ae0862abc44584" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.770170 8419 raft_consensus.cc:2393] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in this term.
I20251028 03:00:00.770483 8220 leader_election.cc:304] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:00.770588 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 11f422fe1c6e437f9f95c328c9c2a000 (DEFAULT_TABLE table=default.Table_37a7da4dda72400d81691a3598906d38 [id=1d2560979b574b319b04eea3fd3348f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.770772 9595 raft_consensus.cc:3060] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.770839 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f422fe1c6e437f9f95c328c9c2a000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.771422 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07d3e6c640434652a010942402c54716" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.771653 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "66d385494c7a4fb8a6501519925be7d4" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.771997 9595 raft_consensus.cc:2749] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.772004 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07d3e6c640434652a010942402c54716" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.772768 8546 tablet_bootstrap.cc:654] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.761730 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 6758750bd7054b38bc3afabd2adaeb32 (DEFAULT_TABLE table=default.Table_9b47e45dc8f8444e8be57929ceef64d4 [id=947344a95a0d476dbb8056855ab25417]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.774264 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6758750bd7054b38bc3afabd2adaeb32. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.774861 8546 tablet_bootstrap.cc:492] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.774940 8546 ts_tablet_manager.cc:1403] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 03:00:00.775425 8546 raft_consensus.cc:359] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.775519 8546 raft_consensus.cc:385] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.775552 8546 raft_consensus.cc:740] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.775671 8546 consensus_queue.cc:260] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.775949 8546 ts_tablet_manager.cc:1434] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.776122 8546 tablet_bootstrap.cc:492] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.776593 9788 raft_consensus.cc:493] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.776691 9788 raft_consensus.cc:515] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:00.776804 8083 leader_election.cc:343] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:00.776985 8083 leader_election.cc:343] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.777060 8546 tablet_bootstrap.cc:654] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.777072 9788 leader_election.cc:290] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:00.777151 8083 leader_election.cc:343] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.777199 8083 leader_election.cc:304] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.777328 9788 raft_consensus.cc:2749] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.777431 9788 consensus_queue.cc:1048] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.778144 8546 tablet_bootstrap.cc:492] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.778223 8546 ts_tablet_manager.cc:1403] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 03:00:00.778384 9639 consensus_queue.cc:1048] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.778656 8546 raft_consensus.cc:359] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.778745 8546 raft_consensus.cc:385] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.778790 8546 raft_consensus.cc:740] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.778900 8546 consensus_queue.cc:260] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.779166 8546 ts_tablet_manager.cc:1434] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 03:00:00.777441 8083 leader_election.cc:343] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.779326 8083 leader_election.cc:304] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.779350 8546 tablet_bootstrap.cc:492] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.779747 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "626c2cc2d19047b794bf16c1b078941f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.780054 9888 raft_consensus.cc:2749] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251028 03:00:00.780251 8083 leader_election.cc:343] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.780401 8546 tablet_bootstrap.cc:654] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.782096 8546 tablet_bootstrap.cc:492] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.782178 8546 ts_tablet_manager.cc:1403] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 03:00:00.782625 8546 raft_consensus.cc:359] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.782720 8546 raft_consensus.cc:385] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.782765 8546 raft_consensus.cc:740] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.782882 8546 consensus_queue.cc:260] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.783156 8546 ts_tablet_manager.cc:1434] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.783344 8546 tablet_bootstrap.cc:492] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.783409 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "626c2cc2d19047b794bf16c1b078941f" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.784312 8546 tablet_bootstrap.cc:654] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:00.785568 8083 leader_election.cc:343] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.785718 8083 leader_election.cc:304] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.786008 9888 raft_consensus.cc:2749] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.787514 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet e0576bc52f99440b9a6e7caf92309bec
I20251028 03:00:00.788023 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_892e6fce625c4e529ee8b9d81330b0d3" } new_table_name: "default.Table_8d72f9b8a87847d0a40f66c2bcdba651" modify_external_catalogs: true
I20251028 03:00:00.789014 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6233be1842804618a5efc76a75cd85f5
I20251028 03:00:00.788956 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7e3855a11e534bf3ba179ca0f177ce70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.789358 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet f6c75f26cb2e46be8b4e427fe9b1b076 from {username='slave'} at 127.0.0.1:47364
W20251028 03:00:00.789377 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e3855a11e534bf3ba179ca0f177ce70 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.789563 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 698cca4847e94a79933c644f6b1c1c8d
I20251028 03:00:00.789783 7963 catalog_manager.cc:6719] ReplaceTablet: tablet 698cca4847e94a79933c644f6b1c1c8d deleted and replaced by tablet 69aaab1d8d0e453b9303ee084826e8eb
I20251028 03:00:00.789964 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c295b736697341e795d7de52b865c4ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.790262 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c295b736697341e795d7de52b865c4ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:00.789014 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d5c3b02db24d4dc48260b42841844938 consensus state has no leader
I20251028 03:00:00.790905 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 6233be1842804618a5efc76a75cd85f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.791065 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 698cca4847e94a79933c644f6b1c1c8d with delete_type TABLET_DATA_DELETED (replaced by tablet 69aaab1d8d0e453b9303ee084826e8eb) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.791152 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c73a1345f4bd40e3b39ca5c4f3cd4fc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.791441 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c73a1345f4bd40e3b39ca5c4f3cd4fc2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:00.791843 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 6233be1842804618a5efc76a75cd85f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6233be1842804618a5efc76a75cd85f5 already in progress: creating tablet
W20251028 03:00:00.792014 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 698cca4847e94a79933c644f6b1c1c8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 698cca4847e94a79933c644f6b1c1c8d already in progress: creating tablet
I20251028 03:00:00.793392 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4de961f542d84fad8cbce64ce99df114
I20251028 03:00:00.793752 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4f4f1df2ece642a5ab6f93331df81ea0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.794072 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f4f1df2ece642a5ab6f93331df81ea0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.795512 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet e0576bc52f99440b9a6e7caf92309bec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.795675 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 6233be1842804618a5efc76a75cd85f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.795799 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 391608f115e845e88d07ae08ea543e8e
I20251028 03:00:00.796051 7967 catalog_manager.cc:6719] ReplaceTablet: tablet 391608f115e845e88d07ae08ea543e8e deleted and replaced by tablet d802f3cffe6a4e0cabbda0fb7fe495bc
I20251028 03:00:00.796205 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f7fc5927d19b467a91d8c9a63e16d275" } modify_external_catalogs: true
I20251028 03:00:00.796294 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f7fc5927d19b467a91d8c9a63e16d275" } modify_external_catalogs: true
I20251028 03:00:00.797142 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 4de961f542d84fad8cbce64ce99df114 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.797341 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 391608f115e845e88d07ae08ea543e8e with delete_type TABLET_DATA_DELETED (replaced by tablet d802f3cffe6a4e0cabbda0fb7fe495bc) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.797650 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 698cca4847e94a79933c644f6b1c1c8d with delete_type TABLET_DATA_DELETED (replaced by tablet 69aaab1d8d0e453b9303ee084826e8eb) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:00.797717 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4de961f542d84fad8cbce64ce99df114 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4de961f542d84fad8cbce64ce99df114 already in progress: creating tablet
W20251028 03:00:00.797865 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 391608f115e845e88d07ae08ea543e8e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 391608f115e845e88d07ae08ea543e8e already in progress: creating tablet
I20251028 03:00:00.798408 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 4de961f542d84fad8cbce64ce99df114 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.799113 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 391608f115e845e88d07ae08ea543e8e with delete_type TABLET_DATA_DELETED (replaced by tablet d802f3cffe6a4e0cabbda0fb7fe495bc) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.799459 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 391608f115e845e88d07ae08ea543e8e with delete_type TABLET_DATA_DELETED (replaced by tablet d802f3cffe6a4e0cabbda0fb7fe495bc) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:00.799922 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 391608f115e845e88d07ae08ea543e8e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 391608f115e845e88d07ae08ea543e8e already in progress: creating tablet
I20251028 03:00:00.800654 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e3ed44682141426f856f581db3e481af" } modify_external_catalogs: true
I20251028 03:00:00.800763 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e3ed44682141426f856f581db3e481af" } modify_external_catalogs: true
I20251028 03:00:00.801672 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet 698cca4847e94a79933c644f6b1c1c8d with delete_type TABLET_DATA_DELETED (replaced by tablet 69aaab1d8d0e453b9303ee084826e8eb) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.801839 9639 consensus_queue.cc:1048] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.801893 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet 6233be1842804618a5efc76a75cd85f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.802019 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 4de961f542d84fad8cbce64ce99df114 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:00.802551 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 698cca4847e94a79933c644f6b1c1c8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 698cca4847e94a79933c644f6b1c1c8d already in progress: creating tablet
W20251028 03:00:00.802709 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 6233be1842804618a5efc76a75cd85f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6233be1842804618a5efc76a75cd85f5 already in progress: creating tablet
I20251028 03:00:00.803551 9888 consensus_queue.cc:1048] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.806159 9888 raft_consensus.cc:493] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.806280 9888 raft_consensus.cc:515] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.807081 9888 leader_election.cc:290] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.807439 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "69e0358b593f4b55b6b4ec859e2c021d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.807935 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "69e0358b593f4b55b6b4ec859e2c021d" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.808264 8083 leader_election.cc:343] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:00.808374 8083 leader_election.cc:343] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.808411 8083 leader_election.cc:304] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
W20251028 03:00:00.808997 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4de961f542d84fad8cbce64ce99df114 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4de961f542d84fad8cbce64ce99df114 already in progress: creating tablet
I20251028 03:00:00.809223 9888 raft_consensus.cc:2749] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.810925 9639 raft_consensus.cc:493] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.811025 9639 raft_consensus.cc:515] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.811442 9639 leader_election.cc:290] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:00.812330 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a38ef10c145a4d00a26265a9dee5c744 consensus state has no leader
I20251028 03:00:00.812593 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffe824a618714776bfab38679c9565e0" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.813226 9888 raft_consensus.cc:493] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251028 03:00:00.813349 8356 leader_election.cc:343] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.813333 9888 raft_consensus.cc:515] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.813735 9888 leader_election.cc:290] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.814162 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9a28ff8c0ee46cb89ba7e28de2cce45" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.814173 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9a28ff8c0ee46cb89ba7e28de2cce45" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:00.814522 8083 leader_election.cc:343] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:00.815492 8083 leader_election.cc:343] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.815562 8083 leader_election.cc:304] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.815694 9888 raft_consensus.cc:2749] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.815992 8550 ts_tablet_manager.cc:1929] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.816128 8550 log.cc:1199] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8192667f14a44ec296d28efa6057a44d
W20251028 03:00:00.816740 8354 leader_election.cc:343] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: ffe824a618714776bfab38679c9565e0
I20251028 03:00:00.816817 8354 leader_election.cc:304] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:00.817173 8550 ts_tablet_manager.cc:1950] T 8192667f14a44ec296d28efa6057a44d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.818060 9639 raft_consensus.cc:2749] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.819890 8545 tablet_bootstrap.cc:492] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.820024 8545 ts_tablet_manager.cc:1403] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.179s user 0.001s sys 0.002s
I20251028 03:00:00.820515 8545 raft_consensus.cc:359] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.820626 8545 raft_consensus.cc:385] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.820662 8545 raft_consensus.cc:740] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.820784 8545 consensus_queue.cc:260] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.821130 8545 ts_tablet_manager.cc:1434] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.821321 8545 tablet_bootstrap.cc:492] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.822381 8545 tablet_bootstrap.cc:654] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.822510 8468 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:00.827347 8545 tablet_bootstrap.cc:492] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.828060 8545 ts_tablet_manager.cc:1403] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 03:00:00.828186 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_debfc8aa483543dcbc3c9e37d7a16db6" } new_table_name: "default.Table_8a0dcd295acc422e855b4fc5403cb7e3" modify_external_catalogs: true
I20251028 03:00:00.828579 8545 raft_consensus.cc:359] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.828686 8545 raft_consensus.cc:385] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.828715 8545 raft_consensus.cc:740] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.828868 8545 consensus_queue.cc:260] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.830421 8545 ts_tablet_manager.cc:1434] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:00.831648 8545 tablet_bootstrap.cc:492] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.832829 8545 tablet_bootstrap.cc:654] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.836601 8332 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:00.837247 8468 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:00.837715 8544 tablet_bootstrap.cc:492] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.837827 8544 ts_tablet_manager.cc:1403] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.121s user 0.002s sys 0.000s
I20251028 03:00:00.838369 8544 raft_consensus.cc:359] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.838479 8544 raft_consensus.cc:385] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.838508 8544 raft_consensus.cc:740] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.838629 8544 consensus_queue.cc:260] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.838936 8544 ts_tablet_manager.cc:1434] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.839260 8546 tablet_bootstrap.cc:492] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.839375 8546 ts_tablet_manager.cc:1403] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.000s
I20251028 03:00:00.839888 8546 raft_consensus.cc:359] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.840004 8546 raft_consensus.cc:385] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.840041 8546 raft_consensus.cc:740] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.840155 8546 consensus_queue.cc:260] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.840467 8546 ts_tablet_manager.cc:1434] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.840664 8546 tablet_bootstrap.cc:492] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.841130 8553 raft_consensus.cc:493] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.841239 8553 raft_consensus.cc:515] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.841537 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet f28ca2cf41564792b49940aebc9b2c98 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:00.841593 8553 leader_election.cc:290] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.841872 8546 tablet_bootstrap.cc:654] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.842351 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "593ac7984e01478184af49e5b87ceec4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.842497 8145 raft_consensus.cc:2468] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:00.842475 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "593ac7984e01478184af49e5b87ceec4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.842599 8418 raft_consensus.cc:2468] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:00.842954 8218 leader_election.cc:304] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:00.843636 9595 raft_consensus.cc:2804] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.843693 9595 raft_consensus.cc:493] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.843737 9595 raft_consensus.cc:3060] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.844187 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet e0576bc52f99440b9a6e7caf92309bec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.844805 9595 raft_consensus.cc:515] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.845157 9595 leader_election.cc:290] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.845211 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet e0576bc52f99440b9a6e7caf92309bec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.845314 8544 tablet_bootstrap.cc:492] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 03:00:00.845438 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet e0576bc52f99440b9a6e7caf92309bec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e0576bc52f99440b9a6e7caf92309bec already in progress: creating tablet
W20251028 03:00:00.846330 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet e0576bc52f99440b9a6e7caf92309bec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e0576bc52f99440b9a6e7caf92309bec already in progress: creating tablet
I20251028 03:00:00.846410 8544 tablet_bootstrap.cc:654] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.846839 8145 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "593ac7984e01478184af49e5b87ceec4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.846952 8145 raft_consensus.cc:3060] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.847846 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "593ac7984e01478184af49e5b87ceec4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:00.847965 8418 raft_consensus.cc:3060] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.848042 8145 raft_consensus.cc:2468] T 593ac7984e01478184af49e5b87ceec4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:00.848673 8218 leader_election.cc:304] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:00.848951 9595 raft_consensus.cc:2804] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.848997 8418 raft_consensus.cc:2468] T 593ac7984e01478184af49e5b87ceec4 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:00.849035 9595 raft_consensus.cc:697] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:00.849155 9595 consensus_queue.cc:237] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.849676 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8192667f14a44ec296d28efa6057a44d (table default.Table_ef415299e6ef4aa583629acaba69daf0 [id=448356df3cfe4bdcb549b652a04ff06a]) successfully deleted
I20251028 03:00:00.849927 8550 tablet_replica.cc:333] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.850142 8550 raft_consensus.cc:2243] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.850335 8550 raft_consensus.cc:2272] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.850919 8550 ts_tablet_manager.cc:1916] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.853714 8550 ts_tablet_manager.cc:1929] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.853886 8550 log.cc:1199] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/0fcabc4a2fd44043be5451046ff5bd97
I20251028 03:00:00.854198 8550 ts_tablet_manager.cc:1950] T 0fcabc4a2fd44043be5451046ff5bd97 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.854672 8550 tablet_replica.cc:333] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.854738 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 0fcabc4a2fd44043be5451046ff5bd97 (table default.Table_31e973dfaa9b49f2bed3e9dfd11f7ae9 [id=f6bf7ec7e30f4c469ca1adde25fee2d2]) successfully deleted
I20251028 03:00:00.854931 8550 raft_consensus.cc:2243] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.855119 8550 raft_consensus.cc:2272] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.855389 9958 raft_consensus.cc:493] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.855485 9958 raft_consensus.cc:515] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.855858 9958 leader_election.cc:290] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:00.856765 9839 consensus_queue.cc:1048] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.857618 10112 raft_consensus.cc:493] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.857713 10112 raft_consensus.cc:515] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.858433 10112 leader_election.cc:290] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.858765 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2d0dffdf5cc414ab870e0dde75916d6" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.859115 8220 leader_election.cc:343] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 374936c130c841348e0d76cce070a295
W20251028 03:00:00.859160 8083 leader_election.cc:343] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:00.859381 8218 leader_election.cc:343] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 374936c130c841348e0d76cce070a295
I20251028 03:00:00.859443 8218 leader_election.cc:304] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:00.859611 9884 raft_consensus.cc:493] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.859674 9884 raft_consensus.cc:515] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.859871 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2d0dffdf5cc414ab870e0dde75916d6" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.859969 9884 leader_election.cc:290] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.860039 9884 raft_consensus.cc:493] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.860095 9884 raft_consensus.cc:515] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
W20251028 03:00:00.860210 8083 leader_election.cc:343] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.860270 8083 leader_election.cc:304] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.860325 9884 leader_election.cc:290] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.860390 9884 raft_consensus.cc:2749] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.860559 10112 raft_consensus.cc:2749] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.860807 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06f7d43bea27429fb45413e0dc0ba269" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.860865 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00550a40f04046d19e9112814372da6c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:00.860919 8418 raft_consensus.cc:2468] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:00.860986 8421 raft_consensus.cc:2468] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:00.861428 8220 leader_election.cc:304] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.861459 8550 ts_tablet_manager.cc:1916] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.861579 8220 leader_election.cc:304] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.861570 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00550a40f04046d19e9112814372da6c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.861663 8146 raft_consensus.cc:2468] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:00.862015 9884 raft_consensus.cc:2804] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.862066 9884 raft_consensus.cc:493] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.862025 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 597d792ed13941a2a895b0dc305614f1 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:00.862102 9884 raft_consensus.cc:3060] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.863119 9884 raft_consensus.cc:515] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.863528 9884 leader_election.cc:290] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.863548 10112 raft_consensus.cc:493] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.863628 9884 raft_consensus.cc:2804] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.863629 10112 raft_consensus.cc:515] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.863675 9884 raft_consensus.cc:493] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.863714 9884 raft_consensus.cc:3060] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.863951 8545 tablet_bootstrap.cc:492] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.864039 8545 ts_tablet_manager.cc:1403] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20251028 03:00:00.864362 10112 leader_election.cc:290] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.864573 8545 raft_consensus.cc:359] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.864605 9884 raft_consensus.cc:515] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.864678 8545 raft_consensus.cc:385] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.864709 8545 raft_consensus.cc:740] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.864843 8545 consensus_queue.cc:260] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.865147 8545 ts_tablet_manager.cc:1434] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.865196 9884 leader_election.cc:290] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:00.865343 8545 tablet_bootstrap.cc:492] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:00.865532 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba40ba2c066e4c94885a595fd1c4445e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:00.865906 8083 leader_election.cc:343] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.866437 8545 tablet_bootstrap.cc:654] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.866874 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "06f7d43bea27429fb45413e0dc0ba269" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:00.866987 8418 raft_consensus.cc:3060] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.867079 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba40ba2c066e4c94885a595fd1c4445e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:00.867434 8218 leader_election.cc:343] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 06f7d43bea27429fb45413e0dc0ba269
I20251028 03:00:00.867946 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00550a40f04046d19e9112814372da6c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.868034 8418 raft_consensus.cc:2468] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:00.868068 8146 raft_consensus.cc:3060] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.868238 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00550a40f04046d19e9112814372da6c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:00.868321 8420 raft_consensus.cc:3060] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.868988 9839 raft_consensus.cc:493] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.869053 8146 raft_consensus.cc:2468] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:00.869071 9839 raft_consensus.cc:515] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.869272 8420 raft_consensus.cc:2468] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:00.869396 9839 leader_election.cc:290] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.869755 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f6c75f26cb2e46be8b4e427fe9b1b076
I20251028 03:00:00.869822 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8b4b0aa30194ba495adca9138e2454e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.869936 8273 raft_consensus.cc:2468] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:00.870092 7962 catalog_manager.cc:6719] ReplaceTablet: tablet f6c75f26cb2e46be8b4e427fe9b1b076 deleted and replaced by tablet 9da148c6c854499aa0bd234ba2b0a82d
I20251028 03:00:00.870222 8356 leader_election.cc:304] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.870491 9190 raft_consensus.cc:2804] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:00.870543 9190 raft_consensus.cc:493] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:00.870589 9190 raft_consensus.cc:3060] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:00.870596 8083 leader_election.cc:343] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.870666 8083 leader_election.cc:304] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.871068 10112 raft_consensus.cc:2749] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.871058 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_215f8629159d41069edc7ce48ed97ae7" } new_table_name: "default.Table_0bed08535f19450db1c432dd032850e5" modify_external_catalogs: true
I20251028 03:00:00.871423 8220 leader_election.cc:304] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.871595 8220 leader_election.cc:304] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.871570 9190 raft_consensus.cc:515] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.871730 8553 raft_consensus.cc:2804] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.871805 8553 raft_consensus.cc:697] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:00.871898 9190 leader_election.cc:290] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.871971 8553 consensus_queue.cc:237] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.872591 8553 raft_consensus.cc:2804] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.872653 8553 raft_consensus.cc:697] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:00.872740 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8b4b0aa30194ba495adca9138e2454e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:00.872774 8553 consensus_queue.cc:237] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.872838 8273 raft_consensus.cc:3060] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.873752 8273 raft_consensus.cc:2468] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.874034 8356 leader_election.cc:304] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:00.874182 9840 raft_consensus.cc:2804] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:00.874240 9840 raft_consensus.cc:697] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:00.874349 9840 consensus_queue.cc:237] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.874559 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet f6c75f26cb2e46be8b4e427fe9b1b076 with delete_type TABLET_DATA_DELETED (replaced by tablet 9da148c6c854499aa0bd234ba2b0a82d) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.874699 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet f6c75f26cb2e46be8b4e427fe9b1b076 with delete_type TABLET_DATA_DELETED (replaced by tablet 9da148c6c854499aa0bd234ba2b0a82d) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:00.875208 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet f6c75f26cb2e46be8b4e427fe9b1b076 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6c75f26cb2e46be8b4e427fe9b1b076 already in progress: creating tablet
I20251028 03:00:00.875730 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8b4b0aa30194ba495adca9138e2454e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:00.875850 8143 raft_consensus.cc:2468] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:00.875885 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c8b4b0aa30194ba495adca9138e2454e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:00.875988 8142 raft_consensus.cc:3060] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:00.876526 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_713672ff0f77410cb29890c7e63471ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.876925 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_713672ff0f77410cb29890c7e63471ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.877089 8142 raft_consensus.cc:2468] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:00.877743 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet f6c75f26cb2e46be8b4e427fe9b1b076 with delete_type TABLET_DATA_DELETED (replaced by tablet 9da148c6c854499aa0bd234ba2b0a82d) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:00.878310 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f6c75f26cb2e46be8b4e427fe9b1b076 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6c75f26cb2e46be8b4e427fe9b1b076 already in progress: creating tablet
W20251028 03:00:00.878698 8218 leader_election.cc:343] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 06f7d43bea27429fb45413e0dc0ba269
I20251028 03:00:00.879314 9190 consensus_queue.cc:1048] T 923e46aca8284966980ac5b062baf829 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.880432 9190 consensus_queue.cc:1048] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.880604 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 58934e83e36e42e7b913f01b64bd7e70
I20251028 03:00:00.881197 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 58934e83e36e42e7b913f01b64bd7e70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.882189 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 58934e83e36e42e7b913f01b64bd7e70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.882431 8552 tablet_replica.cc:333] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.882443 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 58934e83e36e42e7b913f01b64bd7e70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.882593 8552 raft_consensus.cc:2243] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:00.882659 8551 tablet_replica.cc:333] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.882817 8551 raft_consensus.cc:2243] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.882810 8552 raft_consensus.cc:2272] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.882934 8551 raft_consensus.cc:2272] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.883222 8552 ts_tablet_manager.cc:1916] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.883360 8551 ts_tablet_manager.cc:1916] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.883424 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c27c2f21d3514e40810f3616b16ce0fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.883792 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c27c2f21d3514e40810f3616b16ce0fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.885623 8551 ts_tablet_manager.cc:1929] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 03:00:00.885687 8551 log.cc:1199] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/58934e83e36e42e7b913f01b64bd7e70
I20251028 03:00:00.885814 8552 ts_tablet_manager.cc:1929] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 03:00:00.885885 8552 log.cc:1199] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/58934e83e36e42e7b913f01b64bd7e70
I20251028 03:00:00.885927 8551 ts_tablet_manager.cc:1950] T 58934e83e36e42e7b913f01b64bd7e70 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.886140 8552 ts_tablet_manager.cc:1950] T 58934e83e36e42e7b913f01b64bd7e70 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.886646 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 58934e83e36e42e7b913f01b64bd7e70 (table default.Table_f7fc5927d19b467a91d8c9a63e16d275 [id=7eb5bb61f05d4d79b43385b5e91c5b2e]) successfully deleted
I20251028 03:00:00.887645 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 58934e83e36e42e7b913f01b64bd7e70 (table default.Table_f7fc5927d19b467a91d8c9a63e16d275 [id=7eb5bb61f05d4d79b43385b5e91c5b2e]) successfully deleted
I20251028 03:00:00.895046 8550 ts_tablet_manager.cc:1929] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.895237 8550 log.cc:1199] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/07d3e6c640434652a010942402c54716
I20251028 03:00:00.895843 8550 ts_tablet_manager.cc:1950] T 07d3e6c640434652a010942402c54716 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:00.896448 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a932ef07b1b9433ca569bbae983a4743"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.896909 8544 tablet_bootstrap.cc:492] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
W20251028 03:00:00.896903 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a932ef07b1b9433ca569bbae983a4743 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.896996 8544 ts_tablet_manager.cc:1403] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.000s
I20251028 03:00:00.897483 8544 raft_consensus.cc:359] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.897585 8544 raft_consensus.cc:385] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.897622 8544 raft_consensus.cc:740] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.897756 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1b436561f0bf44e0b2c680ce30919318
I20251028 03:00:00.897771 8544 consensus_queue.cc:260] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.898098 8544 ts_tablet_manager.cc:1434] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 03:00:00.898219 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_07cff4b56d204f6483cbe27f181cc80a" } new_table_name: "default.Table_544f6086ab4a461cb50f0205a25818d9" modify_external_catalogs: true
I20251028 03:00:00.898923 8546 tablet_bootstrap.cc:492] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.899017 8546 ts_tablet_manager.cc:1403] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.058s user 0.002s sys 0.000s
I20251028 03:00:00.899542 8546 raft_consensus.cc:359] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.899657 8546 raft_consensus.cc:385] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.899686 8546 raft_consensus.cc:740] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.899816 8546 consensus_queue.cc:260] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:00.900105 8546 ts_tablet_manager.cc:1434] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 03:00:00.900146 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c0fbcdef1007456997db9fcf610f1e00 consensus state has no leader
I20251028 03:00:00.900300 8546 tablet_bootstrap.cc:492] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.900941 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f28ca2cf41564792b49940aebc9b2c98
I20251028 03:00:00.901235 7968 catalog_manager.cc:6719] ReplaceTablet: tablet f28ca2cf41564792b49940aebc9b2c98 deleted and replaced by tablet 67f3b694dcc5465ca2c8c5535e7efea6
I20251028 03:00:00.901422 8546 tablet_bootstrap.cc:654] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.902017 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet f28ca2cf41564792b49940aebc9b2c98 with delete_type TABLET_DATA_DELETED (replaced by tablet 67f3b694dcc5465ca2c8c5535e7efea6) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.902395 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet f28ca2cf41564792b49940aebc9b2c98 with delete_type TABLET_DATA_DELETED (replaced by tablet 67f3b694dcc5465ca2c8c5535e7efea6) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.902631 8552 tablet_replica.cc:333] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.902667 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 1b436561f0bf44e0b2c680ce30919318 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.902877 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet f28ca2cf41564792b49940aebc9b2c98 with delete_type TABLET_DATA_DELETED (replaced by tablet 67f3b694dcc5465ca2c8c5535e7efea6) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:00.902796 8552 raft_consensus.cc:2243] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.903009 8552 raft_consensus.cc:2272] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.903151 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 1b436561f0bf44e0b2c680ce30919318 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:00.903286 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 1b436561f0bf44e0b2c680ce30919318 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.903391 8552 ts_tablet_manager.cc:1916] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.905725 8552 ts_tablet_manager.cc:1929] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.905803 8552 log.cc:1199] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/f28ca2cf41564792b49940aebc9b2c98
I20251028 03:00:00.905889 8551 tablet_replica.cc:333] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.906018 8552 ts_tablet_manager.cc:1950] T f28ca2cf41564792b49940aebc9b2c98 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.906054 8551 raft_consensus.cc:2243] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:00.906248 8551 pending_rounds.cc:70] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 03:00:00.906291 8551 pending_rounds.cc:77] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597160300085248 op_type: NO_OP noop_request { }
I20251028 03:00:00.906351 8551 raft_consensus.cc:2889] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:00.906404 8551 raft_consensus.cc:2272] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.906883 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet f28ca2cf41564792b49940aebc9b2c98 (table default.Table_5008c46b0dcb452497e6dbfd59c37b92 [id=a7962ddee0e841c0ad97786634c03a75]) successfully deleted
I20251028 03:00:00.906984 8551 ts_tablet_manager.cc:1916] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.909490 8551 ts_tablet_manager.cc:1929] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:00.909989 8551 log.cc:1199] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/f28ca2cf41564792b49940aebc9b2c98
I20251028 03:00:00.910351 8551 ts_tablet_manager.cc:1950] T f28ca2cf41564792b49940aebc9b2c98 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.911121 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet f28ca2cf41564792b49940aebc9b2c98 (table default.Table_5008c46b0dcb452497e6dbfd59c37b92 [id=a7962ddee0e841c0ad97786634c03a75]) successfully deleted
I20251028 03:00:00.911419 8551 tablet_replica.cc:333] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.911599 8551 raft_consensus.cc:2243] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:00.911818 8551 raft_consensus.cc:2272] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.914073 8552 tablet_replica.cc:333] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:00.914278 8195 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:00.914851 8552 raft_consensus.cc:2243] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.915050 8552 raft_consensus.cc:2272] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.915403 8551 ts_tablet_manager.cc:1916] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.917343 8552 ts_tablet_manager.cc:1916] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.923162 8552 ts_tablet_manager.cc:1929] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:00.924642 8544 tablet_bootstrap.cc:492] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.924825 8552 log.cc:1199] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/1b436561f0bf44e0b2c680ce30919318
I20251028 03:00:00.925377 8552 ts_tablet_manager.cc:1950] T 1b436561f0bf44e0b2c680ce30919318 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:00.925782 8544 tablet_bootstrap.cc:654] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.927517 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 1b436561f0bf44e0b2c680ce30919318 (table default.Table_e3ed44682141426f856f581db3e481af [id=f34cf093333b4460b55dd2a474688449]) successfully deleted
I20251028 03:00:00.924697 8551 ts_tablet_manager.cc:1929] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:00.927942 8551 log.cc:1199] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/1b436561f0bf44e0b2c680ce30919318
I20251028 03:00:00.928396 8551 ts_tablet_manager.cc:1950] T 1b436561f0bf44e0b2c680ce30919318 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.928712 8546 tablet_bootstrap.cc:492] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:00.928830 8546 ts_tablet_manager.cc:1403] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 03:00:00.929352 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 07d3e6c640434652a010942402c54716 (table default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]) successfully deleted
I20251028 03:00:00.929361 8546 raft_consensus.cc:359] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.929477 8546 raft_consensus.cc:385] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.929518 8546 raft_consensus.cc:740] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.929646 8546 consensus_queue.cc:260] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:00.930015 8546 ts_tablet_manager.cc:1434] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.930224 8546 tablet_bootstrap.cc:492] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:00.930258 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 1b436561f0bf44e0b2c680ce30919318 (table default.Table_e3ed44682141426f856f581db3e481af [id=f34cf093333b4460b55dd2a474688449]) successfully deleted
W20251028 03:00:00.930594 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ba40ba2c066e4c94885a595fd1c4445e consensus state has no leader
I20251028 03:00:00.931216 8550 tablet_replica.cc:333] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:00.933188 8550 raft_consensus.cc:2243] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.933699 8550 raft_consensus.cc:2272] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.934139 8550 ts_tablet_manager.cc:1916] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.939146 10112 raft_consensus.cc:493] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:00.939266 10112 raft_consensus.cc:515] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.939687 10112 leader_election.cc:290] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:00.940398 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4817e3b21c474b2589d30ef9823032d2" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:00.940629 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4817e3b21c474b2589d30ef9823032d2" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:00.940846 8083 leader_election.cc:343] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:00.941116 8083 leader_election.cc:343] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:00.941187 8083 leader_election.cc:304] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:00.941383 10112 raft_consensus.cc:2749] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:00.944247 8195 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:00.951421 8546 tablet_bootstrap.cc:654] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.951278 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 2bf100407b034626b4a9f7e2340f9cf0 (table default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262]). Expected version 2 got 0
I20251028 03:00:00.951524 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 13fb883843d141f09aa90e66f9b7bf1f (table default.Table_2547f4c4cdbd4d7f9ec732da77dc545a [id=c9c59047d50a4419a9f16431bce05cd1]). Expected version 1 got 0
I20251028 03:00:00.951655 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 1979906599754f7ba049c7a9b23bf26d (table default.Table_cdab567cf8da46e8a4d7ba2a80e1e2d9 [id=9fcc70dd88724a28b404e9d4109015aa]). Expected version 1 got 0
I20251028 03:00:00.951723 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (table default.Table_93b5aac358764b869e8589ccc023ea0e [id=674f76f9afed4f3f9e2c165e2479d938]). Expected version 1 got 0
I20251028 03:00:00.951826 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 40a0431501d348b19ec89fe692741ce5 (table default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52]). Expected version 3 got 0
I20251028 03:00:00.952493 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 1979906599754f7ba049c7a9b23bf26d (table default.Table_cdab567cf8da46e8a4d7ba2a80e1e2d9 [id=9fcc70dd88724a28b404e9d4109015aa]). Expected version 1 got 0
I20251028 03:00:00.952601 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 13fb883843d141f09aa90e66f9b7bf1f (table default.Table_2547f4c4cdbd4d7f9ec732da77dc545a [id=c9c59047d50a4419a9f16431bce05cd1]). Expected version 1 got 0
I20251028 03:00:00.952682 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 00550a40f04046d19e9112814372da6c (table default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]). Expected version 1 got 0
I20251028 03:00:00.953979 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet a9653fc3236f4e0b812de5a859932587 with delete_type TABLET_DATA_DELETED (replaced by tablet 58268ac590f3437a942dcafd927a5496) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.954195 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 2893b3a6e8854187b837a7e6fef58c4e with delete_type TABLET_DATA_DELETED (replaced by tablet f5e7e0614af04cc2a22130906d062a79) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.954247 8551 tablet_replica.cc:333] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.954497 8551 raft_consensus.cc:2243] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.954567 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 1979906599754f7ba049c7a9b23bf26d (table default.Table_cdab567cf8da46e8a4d7ba2a80e1e2d9 [id=9fcc70dd88724a28b404e9d4109015aa]). Expected version 1 got 0
I20251028 03:00:00.954679 8551 raft_consensus.cc:2272] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.954782 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 00550a40f04046d19e9112814372da6c (table default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]). Expected version 1 got 0
I20251028 03:00:00.954921 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 13fb883843d141f09aa90e66f9b7bf1f (table default.Table_2547f4c4cdbd4d7f9ec732da77dc545a [id=c9c59047d50a4419a9f16431bce05cd1]). Expected version 1 got 0
I20251028 03:00:00.955046 7969 catalog_manager.cc:5649] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:00.955233 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 464e6b3483bc4816bfa844c52fd3ca53 (table default.Table_e95d99935d7d4d94a16f5a826847ce32 [id=71026b1300014ba6a66c3431c05cc86c]). Expected version 1 got 0
I20251028 03:00:00.955410 8551 ts_tablet_manager.cc:1916] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.955744 8544 tablet_bootstrap.cc:492] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.955847 8544 ts_tablet_manager.cc:1403] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.001s
I20251028 03:00:00.956323 8544 raft_consensus.cc:359] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.956432 8544 raft_consensus.cc:385] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.956465 8544 raft_consensus.cc:740] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.956581 8544 consensus_queue.cc:260] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.956854 8544 ts_tablet_manager.cc:1434] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.957034 8544 tablet_bootstrap.cc:492] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.958098 8544 tablet_bootstrap.cc:654] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.958803 9958 consensus_queue.cc:1048] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:00.959810 8551 ts_tablet_manager.cc:1929] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.960054 8551 log.cc:1199] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a9653fc3236f4e0b812de5a859932587
I20251028 03:00:00.960417 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet ffe824a618714776bfab38679c9565e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:00.960498 8551 ts_tablet_manager.cc:1950] T a9653fc3236f4e0b812de5a859932587 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.961644 8259 tablet_service.cc:1505] Processing CreateTablet for tablet b155706df5f147ba8a4a6a363b021545 (DEFAULT_TABLE table=default.Table_62b833a367564e2db98147279f26082d [id=54a2ebc7d14445eb9c04a619cb319396]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.961918 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b155706df5f147ba8a4a6a363b021545. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.961903 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a9653fc3236f4e0b812de5a859932587 (table default.Table_0b7b98e5884a48de833ab1fbbad97f08 [id=fdb289a561dd4c6497173ef43e2e9ccd]) successfully deleted
I20251028 03:00:00.962218 8551 tablet_replica.cc:333] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.962430 8551 raft_consensus.cc:2243] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.962670 8551 raft_consensus.cc:2272] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.964385 8551 ts_tablet_manager.cc:1916] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.964933 8401 tablet_service.cc:1505] Processing CreateTablet for tablet b155706df5f147ba8a4a6a363b021545 (DEFAULT_TABLE table=default.Table_62b833a367564e2db98147279f26082d [id=54a2ebc7d14445eb9c04a619cb319396]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.965247 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b155706df5f147ba8a4a6a363b021545. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.965546 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_620249416bb9473b90b7fb58a6032a1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.965945 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_620249416bb9473b90b7fb58a6032a1b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.966872 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_932eaf48b5614bbfb5d581cfd42f16d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:00.967211 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_932eaf48b5614bbfb5d581cfd42f16d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.969607 8551 ts_tablet_manager.cc:1929] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.969753 8551 log.cc:1199] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2893b3a6e8854187b837a7e6fef58c4e
I20251028 03:00:00.970098 8551 ts_tablet_manager.cc:1950] T 2893b3a6e8854187b837a7e6fef58c4e P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.970414 8121 tablet_service.cc:1505] Processing CreateTablet for tablet b155706df5f147ba8a4a6a363b021545 (DEFAULT_TABLE table=default.Table_62b833a367564e2db98147279f26082d [id=54a2ebc7d14445eb9c04a619cb319396]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.970701 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b155706df5f147ba8a4a6a363b021545. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.971150 8112 tablet_service.cc:1505] Processing CreateTablet for tablet b5e87126939c4d559522e7738a81e1d2 (DEFAULT_TABLE table=default.Table_a5594c4b37154c58a5734fadcaf00972 [id=c412f22dd1ef47bc82feb00066b80d28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.971411 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e87126939c4d559522e7738a81e1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.971719 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2893b3a6e8854187b837a7e6fef58c4e (table default.Table_85b63c813dc94719b07f0e12854b164b [id=49bac53017824831928c69211121863b]) successfully deleted
I20251028 03:00:00.972275 8551 tablet_replica.cc:333] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:00.972312 8258 tablet_service.cc:1505] Processing CreateTablet for tablet f6688010ce9148dc926c302d9c3fbfd7 (DEFAULT_TABLE table=default.Table_b0aec51ddd0a4f348a95bb73c0e6d226 [id=4bb167209d5043238d761a2e63ebe742]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.972589 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6688010ce9148dc926c302d9c3fbfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.972571 8551 raft_consensus.cc:2243] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:00.972735 8551 raft_consensus.cc:2272] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:00.973518 8397 tablet_service.cc:1505] Processing CreateTablet for tablet f6688010ce9148dc926c302d9c3fbfd7 (DEFAULT_TABLE table=default.Table_b0aec51ddd0a4f348a95bb73c0e6d226 [id=4bb167209d5043238d761a2e63ebe742]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.973565 8551 ts_tablet_manager.cc:1916] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:00.973760 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6688010ce9148dc926c302d9c3fbfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.976073 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 44323698800f486990acde8ee2909260 (DEFAULT_TABLE table=default.Table_e2683e2b014947bd87e048f63aa666f1 [id=058a75d461a0497a9268f8edf3506ece]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.976054 8122 tablet_service.cc:1505] Processing CreateTablet for tablet f6688010ce9148dc926c302d9c3fbfd7 (DEFAULT_TABLE table=default.Table_b0aec51ddd0a4f348a95bb73c0e6d226 [id=4bb167209d5043238d761a2e63ebe742]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.976293 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6688010ce9148dc926c302d9c3fbfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.976310 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44323698800f486990acde8ee2909260. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.977960 8551 ts_tablet_manager.cc:1929] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:00.978119 8551 log.cc:1199] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ffe824a618714776bfab38679c9565e0
I20251028 03:00:00.978475 8551 ts_tablet_manager.cc:1950] T ffe824a618714776bfab38679c9565e0 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:00.978765 8259 tablet_service.cc:1505] Processing CreateTablet for tablet b5e87126939c4d559522e7738a81e1d2 (DEFAULT_TABLE table=default.Table_a5594c4b37154c58a5734fadcaf00972 [id=c412f22dd1ef47bc82feb00066b80d28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.978765 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 44323698800f486990acde8ee2909260 (DEFAULT_TABLE table=default.Table_e2683e2b014947bd87e048f63aa666f1 [id=058a75d461a0497a9268f8edf3506ece]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.979023 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44323698800f486990acde8ee2909260. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.979126 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e87126939c4d559522e7738a81e1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.979593 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ffe824a618714776bfab38679c9565e0 (table default.Table_6f0cc2934b86479083615ebdb7ead649 [id=652e99fe1390470387f6f57273e4e48b]) successfully deleted
I20251028 03:00:00.981657 8401 tablet_service.cc:1505] Processing CreateTablet for tablet b5e87126939c4d559522e7738a81e1d2 (DEFAULT_TABLE table=default.Table_a5594c4b37154c58a5734fadcaf00972 [id=c412f22dd1ef47bc82feb00066b80d28]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.981911 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e87126939c4d559522e7738a81e1d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.986299 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 44323698800f486990acde8ee2909260 (DEFAULT_TABLE table=default.Table_e2683e2b014947bd87e048f63aa666f1 [id=058a75d461a0497a9268f8edf3506ece]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:00.986673 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44323698800f486990acde8ee2909260. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:00.991848 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_df4ca62b1a17423d8e824e6eb889a2c4" } modify_external_catalogs: true
I20251028 03:00:00.991868 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_cd1078befff84198beaa6679f94a351c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:00.992005 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_df4ca62b1a17423d8e824e6eb889a2c4" } modify_external_catalogs: true
W20251028 03:00:00.992269 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd1078befff84198beaa6679f94a351c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:00.992762 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_70cf1ece6d88419b952556cc05782215" } modify_external_catalogs: true
I20251028 03:00:00.992885 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_70cf1ece6d88419b952556cc05782215" } modify_external_catalogs: true
I20251028 03:00:00.994885 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e2683e2b014947bd87e048f63aa666f1" } new_table_name: "default.Table_393974d2200042a6a13a918ac716dc4b" modify_external_catalogs: true
I20251028 03:00:00.996272 8544 tablet_bootstrap.cc:492] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:00.996373 8544 ts_tablet_manager.cc:1403] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.000s
I20251028 03:00:00.996881 8544 raft_consensus.cc:359] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.997027 8544 raft_consensus.cc:385] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:00.997071 8544 raft_consensus.cc:740] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:00.997201 8544 consensus_queue.cc:260] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:00.997509 8544 ts_tablet_manager.cc:1434] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:00.997701 8544 tablet_bootstrap.cc:492] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:00.998764 8544 tablet_bootstrap.cc:654] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:00.999374 8545 tablet_bootstrap.cc:492] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:00.999482 8545 ts_tablet_manager.cc:1403] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.134s user 0.002s sys 0.000s
I20251028 03:00:00.999945 8545 raft_consensus.cc:359] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.000056 8545 raft_consensus.cc:385] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.000085 8545 raft_consensus.cc:740] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.000201 8545 consensus_queue.cc:260] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.000475 8545 ts_tablet_manager.cc:1434] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.000680 8545 tablet_bootstrap.cc:492] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.001727 8545 tablet_bootstrap.cc:654] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.006129 10112 raft_consensus.cc:493] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.006235 10112 raft_consensus.cc:515] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.006608 10112 leader_election.cc:290] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:01.006983 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2bf100407b034626b4a9f7e2340f9cf0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:01.007412 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2bf100407b034626b4a9f7e2340f9cf0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:01.007606 8083 leader_election.cc:343] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:01.007979 8083 leader_election.cc:343] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.008060 8083 leader_election.cc:304] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:01.009141 10112 raft_consensus.cc:2749] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.009227 9958 raft_consensus.cc:493] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.009326 9958 raft_consensus.cc:515] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.009737 9958 leader_election.cc:290] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:01.010154 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5155389485d54c1b82c45eb65364620f" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:01.011024 8220 leader_election.cc:343] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.015247 8545 tablet_bootstrap.cc:492] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.015393 8545 ts_tablet_manager.cc:1403] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.001s
W20251028 03:00:01.015841 8218 leader_election.cc:343] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 5155389485d54c1b82c45eb65364620f
I20251028 03:00:01.015933 8218 leader_election.cc:304] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.015959 8545 raft_consensus.cc:359] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.016073 8545 raft_consensus.cc:385] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.016110 8545 raft_consensus.cc:740] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.016178 9884 raft_consensus.cc:493] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.016247 8545 consensus_queue.cc:260] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.016258 9884 raft_consensus.cc:515] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.016541 8545 ts_tablet_manager.cc:1434] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:01.016628 9884 leader_election.cc:290] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.016722 9884 raft_consensus.cc:2749] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.017091 8544 tablet_bootstrap.cc:492] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:01.017187 8544 ts_tablet_manager.cc:1403] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20251028 03:00:01.017728 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5bf9795bd59b42ea997a97c84e1ea111" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.017840 8545 tablet_bootstrap.cc:492] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 03:00:01.018204 8220 leader_election.cc:343] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.018261 8544 raft_consensus.cc:359] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.018374 8544 raft_consensus.cc:385] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.018415 8544 raft_consensus.cc:740] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.018569 8544 consensus_queue.cc:260] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.018904 8544 ts_tablet_manager.cc:1434] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20251028 03:00:01.019006 8218 leader_election.cc:343] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 5bf9795bd59b42ea997a97c84e1ea111
I20251028 03:00:01.019071 8218 leader_election.cc:304] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.019260 9958 raft_consensus.cc:2749] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.019413 8545 tablet_bootstrap.cc:654] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.022471 9958 raft_consensus.cc:493] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.022575 9958 raft_consensus.cc:515] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.022902 9958 leader_election.cc:290] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:01.023783 8220 leader_election.cc:343] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: b9803f4cfb9b4ef9a1954c83985d6258
I20251028 03:00:01.023995 8544 tablet_bootstrap.cc:492] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:01.024179 10112 raft_consensus.cc:493] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.024241 10112 raft_consensus.cc:515] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.024550 10112 leader_election.cc:290] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 03:00:01.024557 8218 leader_election.cc:343] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: b9803f4cfb9b4ef9a1954c83985d6258
I20251028 03:00:01.024622 8218 leader_election.cc:304] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.025713 8544 tablet_bootstrap.cc:654] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.026075 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40a0431501d348b19ec89fe692741ce5" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.026208 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40a0431501d348b19ec89fe692741ce5" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:01.027429 8083 leader_election.cc:343] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.027459 10112 raft_consensus.cc:493] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251028 03:00:01.027598 8083 leader_election.cc:343] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.027658 8083 leader_election.cc:304] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:01.028005 9888 raft_consensus.cc:2749] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.027994 9958 raft_consensus.cc:2749] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.027561 10112 raft_consensus.cc:515] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.028430 10112 leader_election.cc:290] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:01.030658 8550 ts_tablet_manager.cc:1929] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.030740 8550 log.cc:1199] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c8b4b0aa30194ba495adca9138e2454e
I20251028 03:00:01.030970 8550 ts_tablet_manager.cc:1950] T c8b4b0aa30194ba495adca9138e2454e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.031764 8550 tablet_replica.cc:333] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.031939 8550 raft_consensus.cc:2243] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.032076 8550 raft_consensus.cc:2272] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.032475 8550 ts_tablet_manager.cc:1916] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.034018 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f422030a6a4042009efc7fcfb21a6bca" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.034158 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f422030a6a4042009efc7fcfb21a6bca" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:01.036108 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet c8b4b0aa30194ba495adca9138e2454e (table default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]) successfully deleted
W20251028 03:00:01.036285 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ebf449a6d241486a892eba79568c3409 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ebf449a6d241486a892eba79568c3409 already in progress: creating tablet
W20251028 03:00:01.036406 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e0576bc52f99440b9a6e7caf92309bec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e0576bc52f99440b9a6e7caf92309bec already in progress: creating tablet
W20251028 03:00:01.036509 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6233be1842804618a5efc76a75cd85f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6233be1842804618a5efc76a75cd85f5 already in progress: creating tablet
W20251028 03:00:01.036607 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 698cca4847e94a79933c644f6b1c1c8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 698cca4847e94a79933c644f6b1c1c8d already in progress: creating tablet
W20251028 03:00:01.036705 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4de961f542d84fad8cbce64ce99df114 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4de961f542d84fad8cbce64ce99df114 already in progress: creating tablet
W20251028 03:00:01.036835 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 391608f115e845e88d07ae08ea543e8e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 391608f115e845e88d07ae08ea543e8e already in progress: creating tablet
I20251028 03:00:01.037209 8546 tablet_bootstrap.cc:492] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.037300 8546 ts_tablet_manager.cc:1403] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.107s user 0.003s sys 0.000s
I20251028 03:00:01.037883 8546 raft_consensus.cc:359] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.037990 8546 raft_consensus.cc:385] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.038026 8546 raft_consensus.cc:740] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.038168 8546 consensus_queue.cc:260] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.038446 8546 ts_tablet_manager.cc:1434] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.038647 8546 tablet_bootstrap.cc:492] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 03:00:01.039657 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2d0e84daebfe481c9df8459e33b46e16 consensus state has no leader
I20251028 03:00:01.039913 8546 tablet_bootstrap.cc:654] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.039942 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3bc315b20a2c41dead9f9a2385b88949" } modify_external_catalogs: true
I20251028 03:00:01.040059 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3bc315b20a2c41dead9f9a2385b88949" } modify_external_catalogs: true
I20251028 03:00:01.041306 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e29ed40d642b400192ea2dc90e9c28b0" } new_table_name: "default.Table_bc45a4bd94f144cba69034f07727ae37" modify_external_catalogs: true
I20251028 03:00:01.043090 7964 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_d9b8ba18e24848b1b8511a66b8c7cf2f [id=181b50c2d29a4b17b8189e3abb10a420] alter complete (version 2)
W20251028 03:00:01.043576 7964 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet 40a0431501d348b19ec89fe692741ce5 (table default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet 40a0431501d348b19ec89fe692741ce5 consensus state has no leader
I20251028 03:00:01.043808 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 5a3394a1e1254df19e6e3f6641de825a from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.044164 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0c654fd0364e44fa8d063a3c40273316"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.044540 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c654fd0364e44fa8d063a3c40273316 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.046523 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5b4d3ba3f03648b0b3613ccc0c1acc6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.046851 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b4d3ba3f03648b0b3613ccc0c1acc6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.048276 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_13592b2f4e954570b468b492db58d8e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.048578 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13592b2f4e954570b468b492db58d8e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.050866 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 06f7d43bea27429fb45413e0dc0ba269 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.050992 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 06f7d43bea27429fb45413e0dc0ba269 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.051092 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet 2893b3a6e8854187b837a7e6fef58c4e with delete_type TABLET_DATA_DELETED (replaced by tablet f5e7e0614af04cc2a22130906d062a79) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.051151 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 8192667f14a44ec296d28efa6057a44d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.051208 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet c8b4b0aa30194ba495adca9138e2454e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.051259 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet b9803f4cfb9b4ef9a1954c83985d6258 with delete_type TABLET_DATA_DELETED (replaced by tablet 2b6f62b942a94af783c026a043cc60a3) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.051355 8249 tablet_service.cc:1552] Processing DeleteTablet for tablet a9653fc3236f4e0b812de5a859932587 with delete_type TABLET_DATA_DELETED (replaced by tablet 58268ac590f3437a942dcafd927a5496) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.051656 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet c8b4b0aa30194ba495adca9138e2454e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.051709 8552 tablet_replica.cc:333] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.051865 8552 raft_consensus.cc:2243] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.051986 8552 raft_consensus.cc:2272] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.052225 8545 tablet_bootstrap.cc:492] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.052328 8545 ts_tablet_manager.cc:1403] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.002s
I20251028 03:00:01.052542 8552 ts_tablet_manager.cc:1916] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.052819 9884 consensus_queue.cc:1048] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:01.055024 8552 ts_tablet_manager.cc:1929] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.055091 8552 log.cc:1199] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/2893b3a6e8854187b837a7e6fef58c4e
I20251028 03:00:01.056571 8552 ts_tablet_manager.cc:1950] T 2893b3a6e8854187b837a7e6fef58c4e P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.056615 8545 raft_consensus.cc:359] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.056738 8545 raft_consensus.cc:385] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.056775 8545 raft_consensus.cc:740] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.056912 8545 consensus_queue.cc:260] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.057250 8552 tablet_replica.cc:333] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.057301 8545 ts_tablet_manager.cc:1434] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20251028 03:00:01.057509 8545 tablet_bootstrap.cc:492] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.057504 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 2893b3a6e8854187b837a7e6fef58c4e (table default.Table_85b63c813dc94719b07f0e12854b164b [id=49bac53017824831928c69211121863b]) successfully deleted
I20251028 03:00:01.057834 8552 raft_consensus.cc:2243] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.058043 8552 raft_consensus.cc:2272] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.058056 8546 tablet_bootstrap.cc:492] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.058323 8546 ts_tablet_manager.cc:1403] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.002s
I20251028 03:00:01.058661 8545 tablet_bootstrap.cc:654] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.058779 8546 raft_consensus.cc:359] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.058873 8546 raft_consensus.cc:385] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.058902 8546 raft_consensus.cc:740] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.059010 8546 consensus_queue.cc:260] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.059280 8546 ts_tablet_manager.cc:1434] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:01.059480 8546 tablet_bootstrap.cc:492] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:01.060472 8546 tablet_bootstrap.cc:654] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:01.061694 8083 leader_election.cc:343] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:01.061865 8083 leader_election.cc:343] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.061933 8083 leader_election.cc:304] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:01.062091 10112 raft_consensus.cc:2749] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.063052 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 626c2cc2d19047b794bf16c1b078941f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:01.065253 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f6c75f26cb2e46be8b4e427fe9b1b076 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6c75f26cb2e46be8b4e427fe9b1b076 already in progress: creating tablet
I20251028 03:00:01.066227 8552 ts_tablet_manager.cc:1916] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.068945 8552 ts_tablet_manager.cc:1929] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.069077 8552 log.cc:1199] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/8192667f14a44ec296d28efa6057a44d
I20251028 03:00:01.069335 8552 ts_tablet_manager.cc:1950] T 8192667f14a44ec296d28efa6057a44d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.070184 8550 ts_tablet_manager.cc:1929] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 03:00:01.070888 8550 log.cc:1199] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/58934e83e36e42e7b913f01b64bd7e70
I20251028 03:00:01.072250 8552 tablet_replica.cc:333] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.072415 8552 raft_consensus.cc:2243] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.072520 8552 raft_consensus.cc:2272] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.072952 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 8192667f14a44ec296d28efa6057a44d (table default.Table_ef415299e6ef4aa583629acaba69daf0 [id=448356df3cfe4bdcb549b652a04ff06a]) successfully deleted
I20251028 03:00:01.073038 8552 ts_tablet_manager.cc:1916] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.076402 8550 ts_tablet_manager.cc:1950] T 58934e83e36e42e7b913f01b64bd7e70 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.076732 9884 raft_consensus.cc:493] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.076841 9884 raft_consensus.cc:515] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.076885 8550 tablet_replica.cc:333] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.077116 8550 raft_consensus.cc:2243] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.077193 9884 leader_election.cc:290] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:01.077327 8550 raft_consensus.cc:2272] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:01.077910 8220 leader_election.cc:343] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: a9653fc3236f4e0b812de5a859932587
I20251028 03:00:01.077983 8550 ts_tablet_manager.cc:1916] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.080272 8550 ts_tablet_manager.cc:1929] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.080421 8550 log.cc:1199] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/1b436561f0bf44e0b2c680ce30919318
I20251028 03:00:01.080724 8550 ts_tablet_manager.cc:1950] T 1b436561f0bf44e0b2c680ce30919318 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.081307 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 58934e83e36e42e7b913f01b64bd7e70 (table default.Table_f7fc5927d19b467a91d8c9a63e16d275 [id=7eb5bb61f05d4d79b43385b5e91c5b2e]) successfully deleted
W20251028 03:00:01.084882 8218 leader_election.cc:343] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: a9653fc3236f4e0b812de5a859932587
I20251028 03:00:01.084964 8552 ts_tablet_manager.cc:1929] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.084976 8218 leader_election.cc:304] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.085024 8552 log.cc:1199] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c8b4b0aa30194ba495adca9138e2454e
I20251028 03:00:01.085244 8552 ts_tablet_manager.cc:1950] T c8b4b0aa30194ba495adca9138e2454e P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.085269 8553 raft_consensus.cc:2749] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.087352 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 1b436561f0bf44e0b2c680ce30919318 (table default.Table_e3ed44682141426f856f581db3e481af [id=f34cf093333b4460b55dd2a474688449]) successfully deleted
I20251028 03:00:01.087689 8550 tablet_replica.cc:333] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.087882 8550 raft_consensus.cc:2243] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.088052 8550 pending_rounds.cc:70] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 03:00:01.088317 8550 pending_rounds.cc:77] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597160300085248 op_type: NO_OP noop_request { }
I20251028 03:00:01.088451 8550 raft_consensus.cc:2889] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:01.088559 8550 raft_consensus.cc:2272] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.089155 8550 ts_tablet_manager.cc:1916] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.092130 8550 ts_tablet_manager.cc:1929] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.092286 8550 log.cc:1199] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/f28ca2cf41564792b49940aebc9b2c98
I20251028 03:00:01.092615 8550 ts_tablet_manager.cc:1950] T f28ca2cf41564792b49940aebc9b2c98 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.093091 8550 tablet_replica.cc:333] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.093295 8550 raft_consensus.cc:2243] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.093451 8550 raft_consensus.cc:2272] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.094017 8550 ts_tablet_manager.cc:1916] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.094588 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c8b4b0aa30194ba495adca9138e2454e (table default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]) successfully deleted
I20251028 03:00:01.094854 8552 tablet_replica.cc:333] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.096524 8550 ts_tablet_manager.cc:1929] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.096683 8550 log.cc:1199] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/626c2cc2d19047b794bf16c1b078941f
I20251028 03:00:01.096706 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet bef124efe7934e5284441693573e7f40
I20251028 03:00:01.096717 8552 raft_consensus.cc:2243] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.096849 8552 raft_consensus.cc:2272] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.096992 8550 ts_tablet_manager.cc:1950] T 626c2cc2d19047b794bf16c1b078941f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.097479 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet bef124efe7934e5284441693573e7f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.097649 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet bef124efe7934e5284441693573e7f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.098130 7962 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet aa36c6870c1b4a459f0a41c84855a76e (table default.Table_d51065c1d57c4857bc5765d1baf9da06 [id=3815e9736bd34d668ec7f7e113e11e6f]). Expected version 1 got 0
I20251028 03:00:01.098265 8552 ts_tablet_manager.cc:1916] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.098966 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 with delete_type TABLET_DATA_DELETED (replaced by tablet 751777e4aac9489183e021c2367d3122) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.099661 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_326468a8b8bb4cab93e3f3c6bb66843b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.100381 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 44323698800f486990acde8ee2909260 consensus state has no leader
I20251028 03:00:01.100564 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet f1d727c3dbef467da0aec436ffc393be from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.100934 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_aa03ff4bf2694e7eabc52863451cba52" } modify_external_catalogs: true
I20251028 03:00:01.101047 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_aa03ff4bf2694e7eabc52863451cba52" } modify_external_catalogs: true
I20251028 03:00:01.101540 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1fb0f525ded64eb3850ed026fac77aec
I20251028 03:00:01.101873 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 37f2c8db834d4e3e93af1b5e40f1c71e
W20251028 03:00:01.102293 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2851b9c52e144c85a5f6673a8723c289 consensus state has no leader
I20251028 03:00:01.102409 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 1fb0f525ded64eb3850ed026fac77aec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.102488 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_cdab567cf8da46e8a4d7ba2a80e1e2d9" } new_table_name: "default.Table_fcc46f9512d14f0ea2eb69c442c50d69" modify_external_catalogs: true
I20251028 03:00:01.102591 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 37f2c8db834d4e3e93af1b5e40f1c71e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.102612 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 37f2c8db834d4e3e93af1b5e40f1c71e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.104991 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_79ced5936022425c91473064b87986d3" } modify_external_catalogs: true
I20251028 03:00:01.105106 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_79ced5936022425c91473064b87986d3" } modify_external_catalogs: true
I20251028 03:00:01.105576 8552 ts_tablet_manager.cc:1929] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.105772 8552 log.cc:1199] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/b9803f4cfb9b4ef9a1954c83985d6258
I20251028 03:00:01.106159 8552 ts_tablet_manager.cc:1950] T b9803f4cfb9b4ef9a1954c83985d6258 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.107405 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet bef124efe7934e5284441693573e7f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.108717 8552 tablet_replica.cc:333] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.108851 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet b9803f4cfb9b4ef9a1954c83985d6258 (table default.Table_849a72ead23447078b8088823de4288e [id=82ae3d70b0b544d89bd8e0ff1717f396]) successfully deleted
I20251028 03:00:01.108947 8552 raft_consensus.cc:2243] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.109098 8552 raft_consensus.cc:2272] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.109877 8552 ts_tablet_manager.cc:1916] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.111272 7969 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet b9b5fb938357499b8888ba29d23612bd (table default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]). Expected version 1 got 0
I20251028 03:00:01.112174 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bf9795bd59b42ea997a97c84e1ea111 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.113924 8552 ts_tablet_manager.cc:1929] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.114140 8552 log.cc:1199] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/a9653fc3236f4e0b812de5a859932587
W20251028 03:00:01.116240 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_326468a8b8bb4cab93e3f3c6bb66843b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.106482 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet f28ca2cf41564792b49940aebc9b2c98 (table default.Table_5008c46b0dcb452497e6dbfd59c37b92 [id=a7962ddee0e841c0ad97786634c03a75]) successfully deleted
I20251028 03:00:01.117069 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 1fb0f525ded64eb3850ed026fac77aec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.117153 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 626c2cc2d19047b794bf16c1b078941f (table default.Table_797c6e55c14f415589c745d9bedfcb07 [id=81dbfb2d58ab45958553cf73e259cdc0]) successfully deleted
W20251028 03:00:01.117267 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet bef124efe7934e5284441693573e7f40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bef124efe7934e5284441693573e7f40 already in progress: creating tablet
I20251028 03:00:01.117976 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 37f2c8db834d4e3e93af1b5e40f1c71e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.118131 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 1fb0f525ded64eb3850ed026fac77aec with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.116891 8552 ts_tablet_manager.cc:1950] T a9653fc3236f4e0b812de5a859932587 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 03:00:01.118644 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 37f2c8db834d4e3e93af1b5e40f1c71e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37f2c8db834d4e3e93af1b5e40f1c71e already in progress: creating tablet
W20251028 03:00:01.118800 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1fb0f525ded64eb3850ed026fac77aec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1fb0f525ded64eb3850ed026fac77aec already in progress: creating tablet
I20251028 03:00:01.119426 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet a9653fc3236f4e0b812de5a859932587 (table default.Table_0b7b98e5884a48de833ab1fbbad97f08 [id=fdb289a561dd4c6497173ef43e2e9ccd]) successfully deleted
I20251028 03:00:01.119653 8552 tablet_replica.cc:333] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.119819 8552 raft_consensus.cc:2243] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.120343 8552 pending_rounds.cc:70] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 03:00:01.120433 8552 pending_rounds.cc:77] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597161973157888 op_type: NO_OP noop_request { }
I20251028 03:00:01.120545 8552 raft_consensus.cc:2889] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:01.120980 8552 raft_consensus.cc:2272] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.121919 8552 ts_tablet_manager.cc:1916] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.122524 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d5c3b02db24d4dc48260b42841844938
I20251028 03:00:01.123577 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet d5c3b02db24d4dc48260b42841844938 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.123741 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet d5c3b02db24d4dc48260b42841844938 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:01.125447 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet d5c3b02db24d4dc48260b42841844938 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d5c3b02db24d4dc48260b42841844938 already in progress: creating tablet
I20251028 03:00:01.126920 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet d5c3b02db24d4dc48260b42841844938 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.127183 7963 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 1979906599754f7ba049c7a9b23bf26d (table default.Table_fcc46f9512d14f0ea2eb69c442c50d69 [id=9fcc70dd88724a28b404e9d4109015aa]). Expected version 2 got 0
I20251028 03:00:01.128213 8552 ts_tablet_manager.cc:1929] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.128351 8552 log.cc:1199] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/06f7d43bea27429fb45413e0dc0ba269
I20251028 03:00:01.128443 8468 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:01.128695 8552 ts_tablet_manager.cc:1950] T 06f7d43bea27429fb45413e0dc0ba269 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.131793 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2547f4c4cdbd4d7f9ec732da77dc545a" } new_table_name: "default.Table_29d2d7fc0d094968b83e38195008a440" modify_external_catalogs: true
I20251028 03:00:01.134482 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet f2d0dffdf5cc414ab870e0dde75916d6 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.136654 9906 tablet.cc:1722] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:01.142911 9909 tablet.cc:1722] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:01.150241 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_934fd45916ce4d068d19c1329ac0836b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.150422 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7cc757dea1fe4568957409286675204d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.150660 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_934fd45916ce4d068d19c1329ac0836b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:01.150767 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7cc757dea1fe4568957409286675204d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.163053 8551 tablet_replica.cc:333] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.163223 8551 raft_consensus.cc:2243] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.163363 8551 raft_consensus.cc:2272] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.164186 9913 tablet.cc:1722] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:01.164930 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 06f7d43bea27429fb45413e0dc0ba269 (table default.Table_b001b0dab18e4d0a8a4421ddf1f836d9 [id=ee6840ec2cd44d6883833d367999752e]) successfully deleted
W20251028 03:00:01.165124 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet bef124efe7934e5284441693573e7f40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bef124efe7934e5284441693573e7f40 already in progress: creating tablet
W20251028 03:00:01.165227 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 37f2c8db834d4e3e93af1b5e40f1c71e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37f2c8db834d4e3e93af1b5e40f1c71e already in progress: creating tablet
W20251028 03:00:01.165311 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 1fb0f525ded64eb3850ed026fac77aec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1fb0f525ded64eb3850ed026fac77aec already in progress: creating tablet
W20251028 03:00:01.165431 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet d5c3b02db24d4dc48260b42841844938 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d5c3b02db24d4dc48260b42841844938 already in progress: creating tablet
I20251028 03:00:01.165649 8544 tablet_bootstrap.cc:492] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:01.165745 8544 ts_tablet_manager.cc:1403] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.142s user 0.002s sys 0.000s
I20251028 03:00:01.166245 8544 raft_consensus.cc:359] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.166356 8544 raft_consensus.cc:385] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.166383 8544 raft_consensus.cc:740] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.166446 8546 tablet_bootstrap.cc:492] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.166491 8544 consensus_queue.cc:260] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.166531 8546 ts_tablet_manager.cc:1403] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.107s user 0.000s sys 0.002s
I20251028 03:00:01.166742 8544 ts_tablet_manager.cc:1434] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:01.166915 8544 tablet_bootstrap.cc:492] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:01.167007 8546 raft_consensus.cc:359] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.167124 8546 raft_consensus.cc:385] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.167155 8546 raft_consensus.cc:740] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.167258 8546 consensus_queue.cc:260] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.167582 8546 ts_tablet_manager.cc:1434] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.167744 8546 tablet_bootstrap.cc:492] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:01.168565 8544 tablet_bootstrap.cc:654] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.168690 8546 tablet_bootstrap.cc:654] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.170006 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet de0ba9623bbc4d09abc7a3571046cdce from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.170629 8545 tablet_bootstrap.cc:492] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.170722 8545 ts_tablet_manager.cc:1403] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.113s user 0.000s sys 0.002s
I20251028 03:00:01.171203 9595 raft_consensus.cc:493] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.171228 8545 raft_consensus.cc:359] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.171351 8545 raft_consensus.cc:385] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.171285 9595 raft_consensus.cc:515] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.171383 8545 raft_consensus.cc:740] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.171496 8545 consensus_queue.cc:260] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.171669 9595 leader_election.cc:290] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.171752 8545 ts_tablet_manager.cc:1434] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.171952 8545 tablet_bootstrap.cc:492] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.172853 9913 tablet.cc:1722] T 13fb883843d141f09aa90e66f9b7bf1f P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:01.172917 8545 tablet_bootstrap.cc:654] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:01.177265 8220 leader_election.cc:343] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: ffe824a618714776bfab38679c9565e0
I20251028 03:00:01.177513 9906 tablet.cc:1722] T 13fb883843d141f09aa90e66f9b7bf1f P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:01.177536 8420 raft_consensus.cc:1275] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:01.180984 9595 consensus_queue.cc:1048] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:01.182561 8546 tablet_bootstrap.cc:492] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.182657 8546 ts_tablet_manager.cc:1403] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.001s
I20251028 03:00:01.183152 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 13de247dfb18478daac9ca5601220faa from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.183151 8546 raft_consensus.cc:359] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.183245 8546 raft_consensus.cc:385] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.183272 8546 raft_consensus.cc:740] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.183435 8546 consensus_queue.cc:260] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.183741 8546 ts_tablet_manager.cc:1434] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.183917 8546 tablet_bootstrap.cc:492] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:01.185005 8546 tablet_bootstrap.cc:654] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.186121 8551 ts_tablet_manager.cc:1916] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.187076 9884 raft_consensus.cc:493] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.187155 9884 raft_consensus.cc:515] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.187530 9884 leader_election.cc:290] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.188134 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "66d385494c7a4fb8a6501519925be7d4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.188258 8418 raft_consensus.cc:2468] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.189626 8220 leader_election.cc:304] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:01.189826 9884 raft_consensus.cc:2804] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:01.189880 9884 raft_consensus.cc:493] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:01.189916 9884 raft_consensus.cc:3060] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.190680 8551 ts_tablet_manager.cc:1929] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.190755 8551 log.cc:1199] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/06f7d43bea27429fb45413e0dc0ba269
I20251028 03:00:01.190908 9884 raft_consensus.cc:515] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.190985 8551 ts_tablet_manager.cc:1950] T 06f7d43bea27429fb45413e0dc0ba269 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.191282 9884 leader_election.cc:290] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.191624 8551 tablet_replica.cc:333] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.191730 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "66d385494c7a4fb8a6501519925be7d4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:01.191762 8551 raft_consensus.cc:2243] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.191823 8418 raft_consensus.cc:3060] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.191838 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 06f7d43bea27429fb45413e0dc0ba269 (table default.Table_b001b0dab18e4d0a8a4421ddf1f836d9 [id=ee6840ec2cd44d6883833d367999752e]) successfully deleted
I20251028 03:00:01.191928 8551 pending_rounds.cc:70] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 03:00:01.191960 8551 pending_rounds.cc:77] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597161982537728 op_type: NO_OP noop_request { }
I20251028 03:00:01.192010 8551 raft_consensus.cc:2889] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:01.192061 8551 raft_consensus.cc:2272] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.192711 8551 ts_tablet_manager.cc:1916] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.192778 8418 raft_consensus.cc:2468] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:01.193822 8220 leader_election.cc:304] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:01.194058 9958 raft_consensus.cc:2804] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:01.194142 9958 raft_consensus.cc:697] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:01.194273 9958 consensus_queue.cc:237] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.194706 8551 ts_tablet_manager.cc:1929] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.194766 8551 log.cc:1199] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c8b4b0aa30194ba495adca9138e2454e
I20251028 03:00:01.194830 10112 raft_consensus.cc:493] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.194928 10112 raft_consensus.cc:515] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.195013 8551 ts_tablet_manager.cc:1950] T c8b4b0aa30194ba495adca9138e2454e P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.195365 10112 leader_election.cc:290] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:01.195672 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet c8b4b0aa30194ba495adca9138e2454e (table default.Table_b1c390b850f444aca6a3856952225164 [id=279734691fc04fe2bdfa1eae41ef31ff]) successfully deleted
I20251028 03:00:01.196127 9840 raft_consensus.cc:493] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.196204 9840 raft_consensus.cc:515] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.196507 9840 leader_election.cc:290] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.196938 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa36c6870c1b4a459f0a41c84855a76e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:01.197047 8273 raft_consensus.cc:2468] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:01.197491 8356 leader_election.cc:304] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:01.197574 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8665d2e933fa423d81a1b32f2ce8629b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.197738 9840 raft_consensus.cc:2804] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:01.197801 9840 raft_consensus.cc:493] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20251028 03:00:01.197777 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet bef124efe7934e5284441693573e7f40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bef124efe7934e5284441693573e7f40 already in progress: creating tablet
I20251028 03:00:01.197844 9840 raft_consensus.cc:3060] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.198055 8551 tablet_replica.cc:333] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.198223 8551 raft_consensus.cc:2243] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.198356 8551 raft_consensus.cc:2272] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.198789 9840 raft_consensus.cc:515] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.198957 8551 ts_tablet_manager.cc:1916] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.199155 9840 leader_election.cc:290] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.199450 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa36c6870c1b4a459f0a41c84855a76e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:01.199550 8273 raft_consensus.cc:3060] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:01.199775 8083 leader_election.cc:343] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.200089 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8665d2e933fa423d81a1b32f2ce8629b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:01.200448 8083 leader_election.cc:343] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.200477 8273 raft_consensus.cc:2468] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:01.200522 8083 leader_election.cc:304] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:01.200670 10112 raft_consensus.cc:2749] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.200892 8356 leader_election.cc:304] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:01.201112 9190 raft_consensus.cc:2804] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:01.201208 9190 raft_consensus.cc:697] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:01.201323 9190 consensus_queue.cc:237] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.202212 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7f9762941e0044468dc41a8d0974ba58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.202611 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f9762941e0044468dc41a8d0974ba58 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.203773 8551 ts_tablet_manager.cc:1929] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.203924 8551 log.cc:1199] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/fa70b61f9c8840b8b8f1b0f0a2ec0a17
I20251028 03:00:01.204200 8551 ts_tablet_manager.cc:1950] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.204180 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa36c6870c1b4a459f0a41c84855a76e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:01.204285 8142 raft_consensus.cc:3060] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.204954 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 (table default.Table_71dd113f5d12423d9aca617205523aa1 [id=9cd25ed7b9a04064b29a42f6fcd95e00]) successfully deleted
I20251028 03:00:01.205260 8142 raft_consensus.cc:2468] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:01.204262 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "aa36c6870c1b4a459f0a41c84855a76e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:01.205432 8146 raft_consensus.cc:2376] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1. Re-sending same reply.
W20251028 03:00:01.205461 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 1fb0f525ded64eb3850ed026fac77aec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1fb0f525ded64eb3850ed026fac77aec already in progress: creating tablet
W20251028 03:00:01.205880 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 37f2c8db834d4e3e93af1b5e40f1c71e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37f2c8db834d4e3e93af1b5e40f1c71e already in progress: creating tablet
I20251028 03:00:01.206094 8551 tablet_replica.cc:333] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.206257 8551 raft_consensus.cc:2243] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.206390 8551 raft_consensus.cc:2272] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.206918 8551 ts_tablet_manager.cc:1916] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.209686 8551 ts_tablet_manager.cc:1929] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.209873 8551 log.cc:1199] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/5bf9795bd59b42ea997a97c84e1ea111
I20251028 03:00:01.210136 8551 ts_tablet_manager.cc:1950] T 5bf9795bd59b42ea997a97c84e1ea111 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.211594 8545 tablet_bootstrap.cc:492] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.211684 8545 ts_tablet_manager.cc:1403] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.000s
I20251028 03:00:01.212208 8545 raft_consensus.cc:359] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.212324 8545 raft_consensus.cc:385] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.212360 8545 raft_consensus.cc:740] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.212472 8545 consensus_queue.cc:260] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.212754 8545 ts_tablet_manager.cc:1434] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.212941 8545 tablet_bootstrap.cc:492] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.213375 9884 raft_consensus.cc:493] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.213539 9884 raft_consensus.cc:515] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:01.214059 9884 leader_election.cc:290] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.214435 7968 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_fcc46f9512d14f0ea2eb69c442c50d69 [id=9fcc70dd88724a28b404e9d4109015aa] alter complete (version 2)
I20251028 03:00:01.214533 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 5bf9795bd59b42ea997a97c84e1ea111 (table default.Table_7bfb43f76f8b46b292deee50a8f58280 [id=473a7ccabea349c087076b5d741690f0]) successfully deleted
I20251028 03:00:01.214581 8545 tablet_bootstrap.cc:654] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:01.215015 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet d5c3b02db24d4dc48260b42841844938 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d5c3b02db24d4dc48260b42841844938 already in progress: creating tablet
I20251028 03:00:01.216198 8545 tablet_bootstrap.cc:492] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.216337 8545 ts_tablet_manager.cc:1403] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251028 03:00:01.215018 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 13de247dfb18478daac9ca5601220faa
I20251028 03:00:01.216751 7961 catalog_manager.cc:6719] ReplaceTablet: tablet 13de247dfb18478daac9ca5601220faa deleted and replaced by tablet f9951c153df74554aba611aa4be7c645
I20251028 03:00:01.217408 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 13de247dfb18478daac9ca5601220faa with delete_type TABLET_DATA_DELETED (replaced by tablet f9951c153df74554aba611aa4be7c645) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.217471 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 13de247dfb18478daac9ca5601220faa with delete_type TABLET_DATA_DELETED (replaced by tablet f9951c153df74554aba611aa4be7c645) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.217584 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 13de247dfb18478daac9ca5601220faa with delete_type TABLET_DATA_DELETED (replaced by tablet f9951c153df74554aba611aa4be7c645) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.217760 8550 tablet_replica.cc:333] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.217917 8550 raft_consensus.cc:2243] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
W20251028 03:00:01.217921 8220 leader_election.cc:343] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: fa70b61f9c8840b8b8f1b0f0a2ec0a17
I20251028 03:00:01.218147 8550 raft_consensus.cc:2272] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.218331 8545 raft_consensus.cc:359] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.218497 8545 raft_consensus.cc:385] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.218573 8545 raft_consensus.cc:740] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.218617 8550 ts_tablet_manager.cc:1916] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.218772 8545 consensus_queue.cc:260] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.219179 8545 ts_tablet_manager.cc:1434] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 03:00:01.220615 8545 tablet_bootstrap.cc:492] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.220620 8146 raft_consensus.cc:1275] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:01.220908 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "66d385494c7a4fb8a6501519925be7d4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:01.221036 8146 raft_consensus.cc:2468] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.221197 8550 ts_tablet_manager.cc:1929] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.221310 8550 log.cc:1199] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/13de247dfb18478daac9ca5601220faa
I20251028 03:00:01.221392 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "66d385494c7a4fb8a6501519925be7d4" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:01.221487 8146 raft_consensus.cc:3060] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.221617 8550 ts_tablet_manager.cc:1950] T 13de247dfb18478daac9ca5601220faa P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.220815 8551 tablet_replica.cc:333] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.222223 8552 tablet_replica.cc:333] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.222215 8551 raft_consensus.cc:2243] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.222362 8552 raft_consensus.cc:2243] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.222324 8551 raft_consensus.cc:2272] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.222437 8146 raft_consensus.cc:2468] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:01.222465 8552 raft_consensus.cc:2272] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.222765 8551 ts_tablet_manager.cc:1916] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.222779 8552 ts_tablet_manager.cc:1916] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.223361 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 13de247dfb18478daac9ca5601220faa (table default.Table_f4eb60b8380645758ce2cadfaff1330a [id=e5dbece40326467088d7435702d8acbf]) successfully deleted
W20251028 03:00:01.225131 8218 leader_election.cc:343] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: ffe824a618714776bfab38679c9565e0
I20251028 03:00:01.225276 8218 leader_election.cc:304] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.225472 8552 ts_tablet_manager.cc:1929] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.225553 8552 log.cc:1199] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/13de247dfb18478daac9ca5601220faa
I20251028 03:00:01.225665 9884 raft_consensus.cc:2749] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.225821 8552 ts_tablet_manager.cc:1950] T 13de247dfb18478daac9ca5601220faa P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 03:00:01.226154 8218 leader_election.cc:343] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: fa70b61f9c8840b8b8f1b0f0a2ec0a17
I20251028 03:00:01.226307 9884 consensus_queue.cc:1048] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:01.226560 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 13de247dfb18478daac9ca5601220faa (table default.Table_f4eb60b8380645758ce2cadfaff1330a [id=e5dbece40326467088d7435702d8acbf]) successfully deleted
I20251028 03:00:01.226652 8218 leader_election.cc:304] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.223546 8545 tablet_bootstrap.cc:654] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.226912 9884 raft_consensus.cc:2749] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.228193 8551 ts_tablet_manager.cc:1929] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.228281 8551 log.cc:1199] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/13de247dfb18478daac9ca5601220faa
I20251028 03:00:01.228567 8551 ts_tablet_manager.cc:1950] T 13de247dfb18478daac9ca5601220faa P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.229053 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet b9b5fb938357499b8888ba29d23612bd (table default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]). Expected version 1 got 0
I20251028 03:00:01.229310 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 13de247dfb18478daac9ca5601220faa (table default.Table_f4eb60b8380645758ce2cadfaff1330a [id=e5dbece40326467088d7435702d8acbf]) successfully deleted
I20251028 03:00:01.229315 7966 catalog_manager.cc:5649] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:01.229583 7966 catalog_manager.cc:5649] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:01.229789 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 00550a40f04046d19e9112814372da6c (table default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]). Expected version 1 got 0
I20251028 03:00:01.230057 8545 tablet_bootstrap.cc:492] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.230136 8545 ts_tablet_manager.cc:1403] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 03:00:01.230623 8545 raft_consensus.cc:359] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.230778 8545 raft_consensus.cc:385] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.230859 8545 raft_consensus.cc:740] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.231051 8545 consensus_queue.cc:260] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.231564 8545 ts_tablet_manager.cc:1434] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.231828 8545 tablet_bootstrap.cc:492] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.232899 8545 tablet_bootstrap.cc:654] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.233517 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 9da684e333f641c099d1852847133965 (DEFAULT_TABLE table=default.Table_c27c2f21d3514e40810f3616b16ce0fb [id=144240473f934b73be4b9e6056668b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.233775 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da684e333f641c099d1852847133965. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.234275 9909 tablet.cc:1722] T 13fb883843d141f09aa90e66f9b7bf1f P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:01.236519 8249 tablet_service.cc:1505] Processing CreateTablet for tablet cc67e4744b814540b79c07a20b6a3926 (DEFAULT_TABLE table=default.Table_713672ff0f77410cb29890c7e63471ad [id=861be5a62d384e459657d028408a5300]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.236795 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc67e4744b814540b79c07a20b6a3926. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.236761 8121 tablet_service.cc:1505] Processing CreateTablet for tablet c4c9d45acb8f4889a48bcff2df52ff82 (DEFAULT_TABLE table=default.Table_4f4f1df2ece642a5ab6f93331df81ea0 [id=9002d531339b46cba80f5317eee54ed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.236914 8112 tablet_service.cc:1505] Processing CreateTablet for tablet cc67e4744b814540b79c07a20b6a3926 (DEFAULT_TABLE table=default.Table_713672ff0f77410cb29890c7e63471ad [id=861be5a62d384e459657d028408a5300]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.236953 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 9da684e333f641c099d1852847133965 (DEFAULT_TABLE table=default.Table_c27c2f21d3514e40810f3616b16ce0fb [id=144240473f934b73be4b9e6056668b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.237013 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c9d45acb8f4889a48bcff2df52ff82. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.237125 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc67e4744b814540b79c07a20b6a3926. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.237169 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da684e333f641c099d1852847133965. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.239070 8251 tablet_service.cc:1505] Processing CreateTablet for tablet c4c9d45acb8f4889a48bcff2df52ff82 (DEFAULT_TABLE table=default.Table_4f4f1df2ece642a5ab6f93331df81ea0 [id=9002d531339b46cba80f5317eee54ed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.239068 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 5af63f43d2f64606ab1becacf75367c9 (DEFAULT_TABLE table=default.Table_c73a1345f4bd40e3b39ca5c4f3cd4fc2 [id=b3f8e98592fa43ebbcc874e8b0be6c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.239194 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 635f6166690f4452a5cf05f9e9a0f253 (DEFAULT_TABLE table=default.Table_7e3855a11e534bf3ba179ca0f177ce70 [id=f7b87214241143db81c62db477f3489c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.239284 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c9d45acb8f4889a48bcff2df52ff82. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.239284 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af63f43d2f64606ab1becacf75367c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.239424 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635f6166690f4452a5cf05f9e9a0f253. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.239826 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 3117aa39cd974ee4975a5ffc3448eb71 (DEFAULT_TABLE table=default.Table_f08ece6c25eb42df91132036c01aac92 [id=44d13df5ae864f3dac855dfc6e1c7037]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.240023 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3117aa39cd974ee4975a5ffc3448eb71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.241300 8115 tablet_service.cc:1505] Processing CreateTablet for tablet 3dc40426dd084395bede5d6618ea4d95 (DEFAULT_TABLE table=default.Table_07adbc0d0e674d0987e90d4d9bb902c1 [id=26f2c81276d745229e978a381d52d63e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.241335 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 5af63f43d2f64606ab1becacf75367c9 (DEFAULT_TABLE table=default.Table_c73a1345f4bd40e3b39ca5c4f3cd4fc2 [id=b3f8e98592fa43ebbcc874e8b0be6c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.241535 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af63f43d2f64606ab1becacf75367c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.241932 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dc40426dd084395bede5d6618ea4d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.242694 8396 tablet_service.cc:1505] Processing CreateTablet for tablet c4c9d45acb8f4889a48bcff2df52ff82 (DEFAULT_TABLE table=default.Table_4f4f1df2ece642a5ab6f93331df81ea0 [id=9002d531339b46cba80f5317eee54ed3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.242957 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c9d45acb8f4889a48bcff2df52ff82. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.243494 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 635f6166690f4452a5cf05f9e9a0f253 (DEFAULT_TABLE table=default.Table_7e3855a11e534bf3ba179ca0f177ce70 [id=f7b87214241143db81c62db477f3489c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.243492 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 5af63f43d2f64606ab1becacf75367c9 (DEFAULT_TABLE table=default.Table_c73a1345f4bd40e3b39ca5c4f3cd4fc2 [id=b3f8e98592fa43ebbcc874e8b0be6c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.243695 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635f6166690f4452a5cf05f9e9a0f253. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.243695 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5af63f43d2f64606ab1becacf75367c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.245745 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 3117aa39cd974ee4975a5ffc3448eb71 (DEFAULT_TABLE table=default.Table_f08ece6c25eb42df91132036c01aac92 [id=44d13df5ae864f3dac855dfc6e1c7037]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.245767 8390 tablet_service.cc:1505] Processing CreateTablet for tablet 635f6166690f4452a5cf05f9e9a0f253 (DEFAULT_TABLE table=default.Table_7e3855a11e534bf3ba179ca0f177ce70 [id=f7b87214241143db81c62db477f3489c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.245956 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 635f6166690f4452a5cf05f9e9a0f253. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.245956 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3117aa39cd974ee4975a5ffc3448eb71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.247910 8389 tablet_service.cc:1505] Processing CreateTablet for tablet 3117aa39cd974ee4975a5ffc3448eb71 (DEFAULT_TABLE table=default.Table_f08ece6c25eb42df91132036c01aac92 [id=44d13df5ae864f3dac855dfc6e1c7037]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.247957 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 3dc40426dd084395bede5d6618ea4d95 (DEFAULT_TABLE table=default.Table_07adbc0d0e674d0987e90d4d9bb902c1 [id=26f2c81276d745229e978a381d52d63e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.248171 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dc40426dd084395bede5d6618ea4d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.248370 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 3dc40426dd084395bede5d6618ea4d95 (DEFAULT_TABLE table=default.Table_07adbc0d0e674d0987e90d4d9bb902c1 [id=26f2c81276d745229e978a381d52d63e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.236768 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 9da684e333f641c099d1852847133965 (DEFAULT_TABLE table=default.Table_c27c2f21d3514e40810f3616b16ce0fb [id=144240473f934b73be4b9e6056668b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.249049 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da684e333f641c099d1852847133965. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.250247 8401 tablet_service.cc:1505] Processing CreateTablet for tablet cc67e4744b814540b79c07a20b6a3926 (DEFAULT_TABLE table=default.Table_713672ff0f77410cb29890c7e63471ad [id=861be5a62d384e459657d028408a5300]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.251149 8546 tablet_bootstrap.cc:492] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.251236 8546 ts_tablet_manager.cc:1403] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.067s user 0.002s sys 0.000s
I20251028 03:00:01.251797 8546 raft_consensus.cc:359] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.251889 8546 raft_consensus.cc:385] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.251919 8546 raft_consensus.cc:740] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.252040 8546 consensus_queue.cc:260] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.252336 8546 ts_tablet_manager.cc:1434] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.252652 8546 tablet_bootstrap.cc:492] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:01.253675 8546 tablet_bootstrap.cc:654] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.254885 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_80e34b41b16242e6968529b0ae18a3bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.255316 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80e34b41b16242e6968529b0ae18a3bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.255424 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 1822a0f5868041368187c862ae3d1a43 (DEFAULT_TABLE table=default.Table_661c1422da1f48deb0f40ffb22acad0e [id=231cee62235140cda043e63a893fdeab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.255656 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1822a0f5868041368187c862ae3d1a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.256434 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2e728b4bb7394afaa5a9fd93d7f2739e" } new_table_name: "default.Table_f454943605324eec83a70310f675f27f" modify_external_catalogs: true
I20251028 03:00:01.257558 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 (DEFAULT_TABLE table=default.Table_c295b736697341e795d7de52b865c4ab [id=be888691a3a343e1b1498e9498577fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.259753 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 30232298155542ecb41f5a1b19b16ffb
I20251028 03:00:01.261427 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 (DEFAULT_TABLE table=default.Table_c295b736697341e795d7de52b865c4ab [id=be888691a3a343e1b1498e9498577fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.261507 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet de0ba9623bbc4d09abc7a3571046cdce
I20251028 03:00:01.261718 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.261739 7967 catalog_manager.cc:6719] ReplaceTablet: tablet de0ba9623bbc4d09abc7a3571046cdce deleted and replaced by tablet 347e824873f8483d974e85440c70c3d2
I20251028 03:00:01.261914 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet 30232298155542ecb41f5a1b19b16ffb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.262051 8387 tablet_service.cc:1505] Processing CreateTablet for tablet d802f3cffe6a4e0cabbda0fb7fe495bc (DEFAULT_TABLE table=default.Table_c061797d60f14b298c963858268b5e60 [id=b5dc2cf2c06e4088a2ce80219d6d675f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.262221 8392 tablet_service.cc:1505] Processing CreateTablet for tablet 67f3b694dcc5465ca2c8c5535e7efea6 (DEFAULT_TABLE table=default.Table_5008c46b0dcb452497e6dbfd59c37b92 [id=a7962ddee0e841c0ad97786634c03a75]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.262327 8257 tablet_service.cc:1505] Processing CreateTablet for tablet 9da148c6c854499aa0bd234ba2b0a82d (DEFAULT_TABLE table=default.Table_b12ebf9b73404d62b93a0eefa8d10b53 [id=4acbcdeb5d644490a70ae38d044d2c22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.262537 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da148c6c854499aa0bd234ba2b0a82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.262816 7961 catalog_manager.cc:5649] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:01.263031 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet aa36c6870c1b4a459f0a41c84855a76e (table default.Table_d51065c1d57c4857bc5765d1baf9da06 [id=3815e9736bd34d668ec7f7e113e11e6f]). Expected version 1 got 0
I20251028 03:00:01.263147 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 464e6b3483bc4816bfa844c52fd3ca53 (table default.Table_e95d99935d7d4d94a16f5a826847ce32 [id=71026b1300014ba6a66c3431c05cc86c]). Expected version 1 got 0
I20251028 03:00:01.264699 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 5155389485d54c1b82c45eb65364620f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.264845 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet 5bf9795bd59b42ea997a97c84e1ea111 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.264955 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 30232298155542ecb41f5a1b19b16ffb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.265194 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 67f3b694dcc5465ca2c8c5535e7efea6 (DEFAULT_TABLE table=default.Table_5008c46b0dcb452497e6dbfd59c37b92 [id=a7962ddee0e841c0ad97786634c03a75]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.265316 8390 tablet_service.cc:1505] Processing CreateTablet for tablet 69aaab1d8d0e453b9303ee084826e8eb (DEFAULT_TABLE table=default.Table_272a84b9ac3e46c882522d75055385d3 [id=507a066c80f643a9b637d3a8f01e8228]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.265403 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67f3b694dcc5465ca2c8c5535e7efea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.265403 8388 tablet_service.cc:1552] Processing DeleteTablet for tablet de0ba9623bbc4d09abc7a3571046cdce with delete_type TABLET_DATA_DELETED (replaced by tablet 347e824873f8483d974e85440c70c3d2) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.265565 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 1822a0f5868041368187c862ae3d1a43 (DEFAULT_TABLE table=default.Table_661c1422da1f48deb0f40ffb22acad0e [id=231cee62235140cda043e63a893fdeab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.265944 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 1822a0f5868041368187c862ae3d1a43 (DEFAULT_TABLE table=default.Table_661c1422da1f48deb0f40ffb22acad0e [id=231cee62235140cda043e63a893fdeab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.266232 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1822a0f5868041368187c862ae3d1a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.267467 8251 tablet_service.cc:1505] Processing CreateTablet for tablet d802f3cffe6a4e0cabbda0fb7fe495bc (DEFAULT_TABLE table=default.Table_c061797d60f14b298c963858268b5e60 [id=b5dc2cf2c06e4088a2ce80219d6d675f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.267683 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d802f3cffe6a4e0cabbda0fb7fe495bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.269200 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc67e4744b814540b79c07a20b6a3926. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.269834 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 9da148c6c854499aa0bd234ba2b0a82d (DEFAULT_TABLE table=default.Table_b12ebf9b73404d62b93a0eefa8d10b53 [id=4acbcdeb5d644490a70ae38d044d2c22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.269881 8115 tablet_service.cc:1505] Processing CreateTablet for tablet 67f3b694dcc5465ca2c8c5535e7efea6 (DEFAULT_TABLE table=default.Table_5008c46b0dcb452497e6dbfd59c37b92 [id=a7962ddee0e841c0ad97786634c03a75]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.270062 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da148c6c854499aa0bd234ba2b0a82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.270124 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67f3b694dcc5465ca2c8c5535e7efea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.271274 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 06f7d43bea27429fb45413e0dc0ba269 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.271649 8118 tablet_service.cc:1505] Processing CreateTablet for tablet d802f3cffe6a4e0cabbda0fb7fe495bc (DEFAULT_TABLE table=default.Table_c061797d60f14b298c963858268b5e60 [id=b5dc2cf2c06e4088a2ce80219d6d675f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.271916 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d802f3cffe6a4e0cabbda0fb7fe495bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.272131 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet b9b5fb938357499b8888ba29d23612bd (table default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]). Expected version 1 got 0
I20251028 03:00:01.272269 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 69aaab1d8d0e453b9303ee084826e8eb (DEFAULT_TABLE table=default.Table_272a84b9ac3e46c882522d75055385d3 [id=507a066c80f643a9b637d3a8f01e8228]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.272336 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 9a81841c22ae49c0a17931d90407accc (table default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]). Expected version 1 got 0
I20251028 03:00:01.272429 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet f422030a6a4042009efc7fcfb21a6bca (table default.Table_a01151cafd944efcbc12e5f654a7a05a [id=fdd6ecd598f240948882d710fd485da2]). Expected version 2 got 0
I20251028 03:00:01.272434 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 69aaab1d8d0e453b9303ee084826e8eb (DEFAULT_TABLE table=default.Table_272a84b9ac3e46c882522d75055385d3 [id=507a066c80f643a9b637d3a8f01e8228]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.272476 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69aaab1d8d0e453b9303ee084826e8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.272639 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69aaab1d8d0e453b9303ee084826e8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.274211 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 9da148c6c854499aa0bd234ba2b0a82d (DEFAULT_TABLE table=default.Table_b12ebf9b73404d62b93a0eefa8d10b53 [id=4acbcdeb5d644490a70ae38d044d2c22]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.274425 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da148c6c854499aa0bd234ba2b0a82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.274499 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 with delete_type TABLET_DATA_DELETED (replaced by tablet 751777e4aac9489183e021c2367d3122) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.274616 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet ffe824a618714776bfab38679c9565e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.274796 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet de0ba9623bbc4d09abc7a3571046cdce with delete_type TABLET_DATA_DELETED (replaced by tablet 347e824873f8483d974e85440c70c3d2) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.275094 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet de0ba9623bbc4d09abc7a3571046cdce with delete_type TABLET_DATA_DELETED (replaced by tablet 347e824873f8483d974e85440c70c3d2) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.275496 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet f28ca2cf41564792b49940aebc9b2c98 with delete_type TABLET_DATA_DELETED (replaced by tablet 67f3b694dcc5465ca2c8c5535e7efea6) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.275621 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet f2d0dffdf5cc414ab870e0dde75916d6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b0c593a3b96434db9ac42d260fc6997) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.276134 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet 1b436561f0bf44e0b2c680ce30919318 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.278658 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 30232298155542ecb41f5a1b19b16ffb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.278980 8550 tablet_replica.cc:333] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.279145 8550 raft_consensus.cc:2243] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.279259 8550 raft_consensus.cc:2272] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.279484 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dc40426dd084395bede5d6618ea4d95. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.279829 8550 ts_tablet_manager.cc:1916] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.281803 8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3117aa39cd974ee4975a5ffc3448eb71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.282027 8550 ts_tablet_manager.cc:1929] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.282105 8550 log.cc:1199] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/f2d0dffdf5cc414ab870e0dde75916d6
I20251028 03:00:01.282320 8550 ts_tablet_manager.cc:1950] T f2d0dffdf5cc414ab870e0dde75916d6 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 03:00:01.283604 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet de0ba9623bbc4d09abc7a3571046cdce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de0ba9623bbc4d09abc7a3571046cdce already in progress: creating tablet
W20251028 03:00:01.283797 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f28ca2cf41564792b49940aebc9b2c98 because the tablet was not found. No further retry: Not found: Tablet not found: f28ca2cf41564792b49940aebc9b2c98
I20251028 03:00:01.283902 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet f2d0dffdf5cc414ab870e0dde75916d6 (table default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]) successfully deleted
W20251028 03:00:01.284004 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1b436561f0bf44e0b2c680ce30919318 because the tablet was not found. No further retry: Not found: Tablet not found: 1b436561f0bf44e0b2c680ce30919318
W20251028 03:00:01.284125 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 30232298155542ecb41f5a1b19b16ffb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 30232298155542ecb41f5a1b19b16ffb already in progress: creating tablet
I20251028 03:00:01.284497 8552 tablet_replica.cc:333] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.284600 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67f3b694dcc5465ca2c8c5535e7efea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.284698 8552 raft_consensus.cc:2243] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.284809 8552 raft_consensus.cc:2272] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.286806 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d802f3cffe6a4e0cabbda0fb7fe495bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.288887 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.290912 10112 raft_consensus.cc:493] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.291014 10112 raft_consensus.cc:515] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.291433 10112 leader_election.cc:290] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:01.291694 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69aaab1d8d0e453b9303ee084826e8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.292083 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d609dc2fea4c79adf0953dc98ad0af" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:01.292058 9958 consensus_queue.cc:1048] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 03:00:01.292769 8083 leader_election.cc:343] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.293578 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d609dc2fea4c79adf0953dc98ad0af" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:01.293915 8083 leader_election.cc:343] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.293985 8083 leader_election.cc:304] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:01.294137 10112 raft_consensus.cc:2749] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.294221 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1822a0f5868041368187c862ae3d1a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.297241 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 4817e3b21c474b2589d30ef9823032d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.297430 8550 tablet_replica.cc:333] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.297575 8550 raft_consensus.cc:2243] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.297695 8550 raft_consensus.cc:2272] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.298270 8550 ts_tablet_manager.cc:1916] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.298933 8546 tablet_bootstrap.cc:492] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.299024 8546 ts_tablet_manager.cc:1403] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.000s
I20251028 03:00:01.299609 8546 raft_consensus.cc:359] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.299718 8546 raft_consensus.cc:385] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.299750 8546 raft_consensus.cc:740] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.299886 8546 consensus_queue.cc:260] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.300050 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0dc0518b395a459abe39e458996330e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.300184 8546 ts_tablet_manager.cc:1434] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.300390 8546 tablet_bootstrap.cc:492] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 03:00:01.300467 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0dc0518b395a459abe39e458996330e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.300689 8550 ts_tablet_manager.cc:1929] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.300755 8550 log.cc:1199] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4817e3b21c474b2589d30ef9823032d2
I20251028 03:00:01.300993 8550 ts_tablet_manager.cc:1950] T 4817e3b21c474b2589d30ef9823032d2 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.301556 8546 tablet_bootstrap.cc:654] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.301892 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_25b2475406494c06be072da724e5351f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.302081 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 (DEFAULT_TABLE table=default.Table_c295b736697341e795d7de52b865c4ab [id=be888691a3a343e1b1498e9498577fc6]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 03:00:01.302265 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25b2475406494c06be072da724e5351f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.302330 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.302649 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7e3855a11e534bf3ba179ca0f177ce70" } modify_external_catalogs: true
I20251028 03:00:01.303771 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e72dcea543a9406bbf87d5de6e5fd9be" } new_table_name: "default.Table_3f85bca98f84492da9cc6b47c772334d" modify_external_catalogs: true
I20251028 03:00:01.303804 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4817e3b21c474b2589d30ef9823032d2 (table default.Table_326908ab91a545829b75b06816614771 [id=06de831f38b74668aabbef36f08a4a12]) successfully deleted
I20251028 03:00:01.304718 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d9e8b9a5238c4b69ade009e5e8cabde7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.305078 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d9e8b9a5238c4b69ade009e5e8cabde7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:01.308095 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 30232298155542ecb41f5a1b19b16ffb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 30232298155542ecb41f5a1b19b16ffb already in progress: creating tablet
W20251028 03:00:01.309065 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet de0ba9623bbc4d09abc7a3571046cdce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de0ba9623bbc4d09abc7a3571046cdce already in progress: creating tablet
I20251028 03:00:01.309587 8552 ts_tablet_manager.cc:1916] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.310281 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f0dafbe2e0a84541a79cf6c887d59619" } new_table_name: "default.Table_3a3b5d6e921341349a2e4c02f4856164" modify_external_catalogs: true
I20251028 03:00:01.312048 8552 ts_tablet_manager.cc:1929] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.312131 8552 log.cc:1199] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/5155389485d54c1b82c45eb65364620f
I20251028 03:00:01.312345 8552 ts_tablet_manager.cc:1950] T 5155389485d54c1b82c45eb65364620f P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.312477 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7e3855a11e534bf3ba179ca0f177ce70" } modify_external_catalogs: true
I20251028 03:00:01.312952 8552 tablet_replica.cc:333] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.313082 8552 raft_consensus.cc:2243] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.313175 8552 raft_consensus.cc:2272] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.314162 8552 ts_tablet_manager.cc:1916] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.315699 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 5155389485d54c1b82c45eb65364620f (table default.Table_b359d8a7e6724c21a2f0de83ddd841b7 [id=eb0e103661fc4bc7b27bb78fc333fd56]) successfully deleted
I20251028 03:00:01.316673 8552 ts_tablet_manager.cc:1929] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.316826 8552 log.cc:1199] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/5bf9795bd59b42ea997a97c84e1ea111
I20251028 03:00:01.317128 8552 ts_tablet_manager.cc:1950] T 5bf9795bd59b42ea997a97c84e1ea111 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.318075 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 5bf9795bd59b42ea997a97c84e1ea111 (table default.Table_7bfb43f76f8b46b292deee50a8f58280 [id=473a7ccabea349c087076b5d741690f0]) successfully deleted
W20251028 03:00:01.318632 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 30232298155542ecb41f5a1b19b16ffb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 30232298155542ecb41f5a1b19b16ffb already in progress: creating tablet
W20251028 03:00:01.319130 7955 catalog_manager.cc:4952] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 06f7d43bea27429fb45413e0dc0ba269 because the tablet was not found. No further retry: Not found: Tablet not found: 06f7d43bea27429fb45413e0dc0ba269
I20251028 03:00:01.319446 8552 tablet_replica.cc:333] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.319626 8552 raft_consensus.cc:2243] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.319777 8552 raft_consensus.cc:2272] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.320463 8552 ts_tablet_manager.cc:1916] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.321880 9958 consensus_queue.cc:1048] T 593ac7984e01478184af49e5b87ceec4 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:01.323241 8552 ts_tablet_manager.cc:1929] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.325927 8552 log.cc:1199] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/fa70b61f9c8840b8b8f1b0f0a2ec0a17
I20251028 03:00:01.326247 8552 ts_tablet_manager.cc:1950] T fa70b61f9c8840b8b8f1b0f0a2ec0a17 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.327214 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet fa70b61f9c8840b8b8f1b0f0a2ec0a17 (table default.Table_71dd113f5d12423d9aca617205523aa1 [id=9cd25ed7b9a04064b29a42f6fcd95e00]) successfully deleted
I20251028 03:00:01.327855 8552 tablet_replica.cc:333] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.328097 8552 raft_consensus.cc:2243] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.328449 8552 raft_consensus.cc:2272] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.328940 8552 ts_tablet_manager.cc:1916] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.331384 8552 ts_tablet_manager.cc:1929] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.331535 8552 log.cc:1199] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/ffe824a618714776bfab38679c9565e0
I20251028 03:00:01.331969 8552 ts_tablet_manager.cc:1950] T ffe824a618714776bfab38679c9565e0 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.332188 8546 tablet_bootstrap.cc:492] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.332268 8546 ts_tablet_manager.cc:1403] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20251028 03:00:01.332669 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet ffe824a618714776bfab38679c9565e0 (table default.Table_6f0cc2934b86479083615ebdb7ead649 [id=652e99fe1390470387f6f57273e4e48b]) successfully deleted
I20251028 03:00:01.332778 8546 raft_consensus.cc:359] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.332881 8546 raft_consensus.cc:385] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.332913 8546 raft_consensus.cc:740] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.333025 8546 consensus_queue.cc:260] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.333329 8546 ts_tablet_manager.cc:1434] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.333510 8546 tablet_bootstrap.cc:492] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 03:00:01.334239 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet de0ba9623bbc4d09abc7a3571046cdce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de0ba9623bbc4d09abc7a3571046cdce already in progress: creating tablet
I20251028 03:00:01.335348 8546 tablet_bootstrap.cc:654] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.338274 8332 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:01.346625 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet aa36c6870c1b4a459f0a41c84855a76e (table default.Table_d51065c1d57c4857bc5765d1baf9da06 [id=3815e9736bd34d668ec7f7e113e11e6f]). Expected version 1 got 0
I20251028 03:00:01.346853 7967 catalog_manager.cc:5649] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:01.347133 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet c0fbcdef1007456997db9fcf610f1e00 (table default.Table_8a0dcd295acc422e855b4fc5403cb7e3 [id=6e7f7ac227db47c98e3a115dddc9ceff]). Expected version 1 got 0
W20251028 03:00:01.351953 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5c1994484abf4b6fbbc3f47527558cdb consensus state has no leader
I20251028 03:00:01.352190 8545 tablet_bootstrap.cc:492] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.352283 8545 ts_tablet_manager.cc:1403] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.121s user 0.002s sys 0.000s
I20251028 03:00:01.352751 8545 raft_consensus.cc:359] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.352857 8545 raft_consensus.cc:385] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.352888 8545 raft_consensus.cc:740] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.353001 8545 consensus_queue.cc:260] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.353281 8545 ts_tablet_manager.cc:1434] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.353276 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_8985e28c7f9d4c84aed8eed96c94a66c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.353461 8545 tablet_bootstrap.cc:492] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 03:00:01.353675 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8985e28c7f9d4c84aed8eed96c94a66c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.354444 8545 tablet_bootstrap.cc:654] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.357555 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ba3d1f9ba1214c1d8ef3d9f737e91d6e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.357982 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba3d1f9ba1214c1d8ef3d9f737e91d6e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.358734 8545 tablet_bootstrap.cc:492] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.358842 8545 ts_tablet_manager.cc:1403] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20251028 03:00:01.359356 8545 raft_consensus.cc:359] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.359493 8545 raft_consensus.cc:385] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.359567 8545 raft_consensus.cc:740] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.359735 8545 consensus_queue.cc:260] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.360209 8545 ts_tablet_manager.cc:1434] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.360291 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3fed043d81f54a3288c752ba83b12cf6" } modify_external_catalogs: true
I20251028 03:00:01.360392 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3fed043d81f54a3288c752ba83b12cf6" } modify_external_catalogs: true
I20251028 03:00:01.360453 8545 tablet_bootstrap.cc:492] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.361538 8545 tablet_bootstrap.cc:654] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.362401 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 635f6166690f4452a5cf05f9e9a0f253
I20251028 03:00:01.362872 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_89cd288f05a043b1bfc0ac7cf4f54236" } modify_external_catalogs: true
I20251028 03:00:01.362964 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_89cd288f05a043b1bfc0ac7cf4f54236" } modify_external_catalogs: true
I20251028 03:00:01.364246 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 635f6166690f4452a5cf05f9e9a0f253 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.364374 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 635f6166690f4452a5cf05f9e9a0f253 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.364779 8468 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
W20251028 03:00:01.365869 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f62b981b9b1a44bb96bbc90f924c3b64 consensus state has no leader
I20251028 03:00:01.366130 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b30c7f4678f74e759917a8de86282725" } new_table_name: "default.Table_ad34ffad24a2474888602aea3c44fcc0" modify_external_catalogs: true
I20251028 03:00:01.366919 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 635f6166690f4452a5cf05f9e9a0f253 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:01.367538 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 635f6166690f4452a5cf05f9e9a0f253 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 635f6166690f4452a5cf05f9e9a0f253 already in progress: creating tablet
W20251028 03:00:01.369254 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 635f6166690f4452a5cf05f9e9a0f253 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 635f6166690f4452a5cf05f9e9a0f253 already in progress: creating tablet
I20251028 03:00:01.369503 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_60724e0676e54a429b5cd8d525030e29" } modify_external_catalogs: true
I20251028 03:00:01.369678 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_60724e0676e54a429b5cd8d525030e29" } modify_external_catalogs: true
W20251028 03:00:01.371084 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 635f6166690f4452a5cf05f9e9a0f253 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 635f6166690f4452a5cf05f9e9a0f253 already in progress: creating tablet
I20251028 03:00:01.373456 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6de616ea4b53462a81fe546b5cc0279c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.373993 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6de616ea4b53462a81fe546b5cc0279c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.378386 8553 consensus_queue.cc:1048] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:01.391079 9913 tablet.cc:1722] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:01.393942 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 240b7da6f46c4c5ab8087ab1dd6275c5
I20251028 03:00:01.394436 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet 9409f3ee379949809f823dfb7f73643b from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.394923 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.395131 8550 tablet_replica.cc:333] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.395270 8550 raft_consensus.cc:2243] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.395432 8550 raft_consensus.cc:2272] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.395874 8550 ts_tablet_manager.cc:1916] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.396082 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.396741 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet b9b5fb938357499b8888ba29d23612bd from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.397053 9909 tablet.cc:1722] T 900973d832c24b62a4554ae9a219cfd8 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20251028 03:00:01.397521 7954 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet f62b981b9b1a44bb96bbc90f924c3b64 (table default.Table_3f85bca98f84492da9cc6b47c772334d [id=d4eaebd005bc42dd86d7125ef555546f], current schema version=5) failed: Not found: failed to reset TS proxy: No leader found: Tablet f62b981b9b1a44bb96bbc90f924c3b64 consensus state has no leader
I20251028 03:00:01.398253 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ad8b00751973482494b4fc0a7cfdf6f0
I20251028 03:00:01.398435 8550 ts_tablet_manager.cc:1929] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.398504 8550 log.cc:1199] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/240b7da6f46c4c5ab8087ab1dd6275c5
I20251028 03:00:01.399008 8552 tablet_replica.cc:333] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.399173 8552 raft_consensus.cc:2243] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.399319 8552 raft_consensus.cc:2272] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.399717 8552 ts_tablet_manager.cc:1916] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.400429 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_059639c7191541caa3dbb30b333d579f" } new_table_name: "default.Table_46c5b7446ccb408598287b67729fcc6c" modify_external_catalogs: true
I20251028 03:00:01.401896 8552 ts_tablet_manager.cc:1929] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.401969 8552 log.cc:1199] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/240b7da6f46c4c5ab8087ab1dd6275c5
I20251028 03:00:01.402361 8552 ts_tablet_manager.cc:1950] T 240b7da6f46c4c5ab8087ab1dd6275c5 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.403679 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
I20251028 03:00:01.404408 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet ad8b00751973482494b4fc0a7cfdf6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.404736 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 240b7da6f46c4c5ab8087ab1dd6275c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.404878 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet ad8b00751973482494b4fc0a7cfdf6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.404990 8551 tablet_replica.cc:333] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
W20251028 03:00:01.404996 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 63ff5fe6968b47119ef0eabe691dc41c consensus state has no leader
I20251028 03:00:01.405122 8551 raft_consensus.cc:2243] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.405193 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 69c9f986f7004ee6a0b6bba624ae1f8d from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.405315 8551 raft_consensus.cc:2272] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.405690 8551 ts_tablet_manager.cc:1916] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.406663 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet e9857ba66e9e4938abe00b1619a275af
I20251028 03:00:01.398834 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet ad8b00751973482494b4fc0a7cfdf6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.407428 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet e9857ba66e9e4938abe00b1619a275af with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.408591 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet e9857ba66e9e4938abe00b1619a275af with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.409180 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet e9857ba66e9e4938abe00b1619a275af with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.410786 8551 ts_tablet_manager.cc:1929] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.410866 8551 log.cc:1199] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/240b7da6f46c4c5ab8087ab1dd6275c5
I20251028 03:00:01.411120 8551 ts_tablet_manager.cc:1950] T 240b7da6f46c4c5ab8087ab1dd6275c5 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.411665 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a56dbd80138241e084ed7c36f85873c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.412158 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
I20251028 03:00:01.412227 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5f3e79b2b1494fe49fae9db68cd963d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.398756 8550 ts_tablet_manager.cc:1950] T 240b7da6f46c4c5ab8087ab1dd6275c5 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.416159 9958 raft_consensus.cc:493] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.416257 9958 raft_consensus.cc:515] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.416659 9958 leader_election.cc:290] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.417068 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0fbcdef1007456997db9fcf610f1e00" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:01.417196 8144 raft_consensus.cc:2468] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.417486 8218 leader_election.cc:304] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:01.417641 9958 raft_consensus.cc:2804] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:01.417690 9958 raft_consensus.cc:493] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:01.417732 9958 raft_consensus.cc:3060] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.418799 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 240b7da6f46c4c5ab8087ab1dd6275c5 (table default.Table_3fed043d81f54a3288c752ba83b12cf6 [id=ae8a178b437f4df4af15bb82bd7a95ee]) successfully deleted
I20251028 03:00:01.418918 9958 raft_consensus.cc:515] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.419286 9958 leader_election.cc:290] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.419703 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0fbcdef1007456997db9fcf610f1e00" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:01.419831 8144 raft_consensus.cc:3060] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:01.420528 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ad8b00751973482494b4fc0a7cfdf6f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad8b00751973482494b4fc0a7cfdf6f0 already in progress: creating tablet
W20251028 03:00:01.420697 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet e9857ba66e9e4938abe00b1619a275af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9857ba66e9e4938abe00b1619a275af already in progress: creating tablet
I20251028 03:00:01.420866 8144 raft_consensus.cc:2468] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
W20251028 03:00:01.416501 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ad8b00751973482494b4fc0a7cfdf6f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad8b00751973482494b4fc0a7cfdf6f0 already in progress: creating tablet
W20251028 03:00:01.421190 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ad8b00751973482494b4fc0a7cfdf6f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad8b00751973482494b4fc0a7cfdf6f0 already in progress: creating tablet
W20251028 03:00:01.421321 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet e9857ba66e9e4938abe00b1619a275af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9857ba66e9e4938abe00b1619a275af already in progress: creating tablet
W20251028 03:00:01.421444 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet e9857ba66e9e4938abe00b1619a275af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9857ba66e9e4938abe00b1619a275af already in progress: creating tablet
I20251028 03:00:01.421424 8218 leader_election.cc:304] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:01.421859 8553 raft_consensus.cc:2804] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:01.421936 8553 raft_consensus.cc:697] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:01.422065 8553 consensus_queue.cc:237] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.422690 8553 raft_consensus.cc:493] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.422762 8553 raft_consensus.cc:515] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.422979 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet b9b5fb938357499b8888ba29d23612bd
I20251028 03:00:01.423115 8553 leader_election.cc:290] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.423288 7960 catalog_manager.cc:6719] ReplaceTablet: tablet b9b5fb938357499b8888ba29d23612bd deleted and replaced by tablet ca6ca361bde14c8aa060ffa12c7bf452
I20251028 03:00:01.423579 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet b9b5fb938357499b8888ba29d23612bd with delete_type TABLET_DATA_DELETED (replaced by tablet ca6ca361bde14c8aa060ffa12c7bf452) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.423733 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet b9b5fb938357499b8888ba29d23612bd with delete_type TABLET_DATA_DELETED (replaced by tablet ca6ca361bde14c8aa060ffa12c7bf452) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.423764 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9409f3ee379949809f823dfb7f73643b
I20251028 03:00:01.423900 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet b9b5fb938357499b8888ba29d23612bd with delete_type TABLET_DATA_DELETED (replaced by tablet ca6ca361bde14c8aa060ffa12c7bf452) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.424077 7964 catalog_manager.cc:6719] ReplaceTablet: tablet 9409f3ee379949809f823dfb7f73643b deleted and replaced by tablet ca2d2049891643cdb12e6e6a507cb493
I20251028 03:00:01.424582 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 9409f3ee379949809f823dfb7f73643b with delete_type TABLET_DATA_DELETED (replaced by tablet ca2d2049891643cdb12e6e6a507cb493) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:01.424610 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4eaa472029a94052abb49241b7622d0f consensus state has no leader
I20251028 03:00:01.424939 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 9409f3ee379949809f823dfb7f73643b with delete_type TABLET_DATA_DELETED (replaced by tablet ca2d2049891643cdb12e6e6a507cb493) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.425002 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 9409f3ee379949809f823dfb7f73643b with delete_type TABLET_DATA_DELETED (replaced by tablet ca2d2049891643cdb12e6e6a507cb493) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:01.425103 8218 leader_election.cc:343] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 0fcabc4a2fd44043be5451046ff5bd97
W20251028 03:00:01.425292 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f3e79b2b1494fe49fae9db68cd963d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.425482 8552 tablet_replica.cc:333] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.425495 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0fbcdef1007456997db9fcf610f1e00" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.425556 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0fbcdef1007456997db9fcf610f1e00" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:01.425637 8552 raft_consensus.cc:2243] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.425640 8421 raft_consensus.cc:2468] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.425995 8552 raft_consensus.cc:2272] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.426267 8551 tablet_replica.cc:333] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.426407 8551 raft_consensus.cc:2243] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
W20251028 03:00:01.426780 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a56dbd80138241e084ed7c36f85873c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.427028 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0fcabc4a2fd44043be5451046ff5bd97" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.427158 8421 raft_consensus.cc:2468] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.427392 9906 tablet.cc:1722] T 900973d832c24b62a4554ae9a219cfd8 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:01.427907 8220 leader_election.cc:304] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.428092 9958 raft_consensus.cc:2804] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:01.428134 9958 raft_consensus.cc:493] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:01.428169 9958 raft_consensus.cc:3060] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.429184 9958 raft_consensus.cc:515] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.429565 9958 leader_election.cc:290] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.430567 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0fcabc4a2fd44043be5451046ff5bd97" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:01.430670 8419 raft_consensus.cc:3060] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.426510 8551 raft_consensus.cc:2272] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.431692 8419 raft_consensus.cc:2468] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:01.431697 8550 tablet_replica.cc:333] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.433125 8552 ts_tablet_manager.cc:1916] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.433234 8550 raft_consensus.cc:2243] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20251028 03:00:01.434180 8218 leader_election.cc:343] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 0fcabc4a2fd44043be5451046ff5bd97
I20251028 03:00:01.434393 8220 leader_election.cc:304] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.434604 9958 raft_consensus.cc:2804] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:01.434674 9958 raft_consensus.cc:697] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:01.434787 9958 consensus_queue.cc:237] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.435256 8553 raft_consensus.cc:493] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.435380 8553 raft_consensus.cc:515] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.435700 8553 leader_election.cc:290] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:01.436074 8550 raft_consensus.cc:2272] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.436396 8551 ts_tablet_manager.cc:1916] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.438033 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b5e3c89222754eacb6ba4773396d62a9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:01.438164 8144 raft_consensus.cc:2468] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.438449 8218 leader_election.cc:304] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:01.438498 8550 ts_tablet_manager.cc:1916] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.438694 8553 raft_consensus.cc:2804] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:01.438746 8553 raft_consensus.cc:493] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:01.438808 8553 raft_consensus.cc:3060] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.439893 8553 raft_consensus.cc:515] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.440316 8553 leader_election.cc:290] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:01.440865 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b5e3c89222754eacb6ba4773396d62a9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:01.440989 8144 raft_consensus.cc:3060] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.442032 8144 raft_consensus.cc:2468] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:01.442361 8218 leader_election.cc:304] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:01.442612 8553 raft_consensus.cc:2804] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:01.442808 8553 raft_consensus.cc:697] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:01.442960 8553 consensus_queue.cc:237] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.444389 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 69c9f986f7004ee6a0b6bba624ae1f8d
I20251028 03:00:01.444442 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b5e3c89222754eacb6ba4773396d62a9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.444573 8419 raft_consensus.cc:2468] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.444725 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 69c9f986f7004ee6a0b6bba624ae1f8d deleted and replaced by tablet 9ea5e82c09354ea88fd67902625e1599
I20251028 03:00:01.444952 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b5e3c89222754eacb6ba4773396d62a9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:01.445065 8419 raft_consensus.cc:3060] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.445313 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 69c9f986f7004ee6a0b6bba624ae1f8d with delete_type TABLET_DATA_DELETED (replaced by tablet 9ea5e82c09354ea88fd67902625e1599) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.445583 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 69c9f986f7004ee6a0b6bba624ae1f8d with delete_type TABLET_DATA_DELETED (replaced by tablet 9ea5e82c09354ea88fd67902625e1599) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.446099 8419 raft_consensus.cc:2468] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:01.446373 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 69c9f986f7004ee6a0b6bba624ae1f8d with delete_type TABLET_DATA_DELETED (replaced by tablet 9ea5e82c09354ea88fd67902625e1599) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.447052 10112 raft_consensus.cc:493] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.447144 10112 raft_consensus.cc:515] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.447641 10112 leader_election.cc:290] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:01.448164 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a81841c22ae49c0a17931d90407accc" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:01.448634 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a81841c22ae49c0a17931d90407accc" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:01.448653 8083 leader_election.cc:343] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:01.449084 8083 leader_election.cc:343] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.449168 8083 leader_election.cc:304] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:01.449411 10112 raft_consensus.cc:2749] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.450364 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 8665d2e933fa423d81a1b32f2ce8629b (table default.Table_139255bc4fb74fae8fb86563b54826bd [id=dc3721126b09466ab5efdb8bd4b46d89]). Expected version 1 got 0
I20251028 03:00:01.450927 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 464e6b3483bc4816bfa844c52fd3ca53 (table default.Table_e95d99935d7d4d94a16f5a826847ce32 [id=71026b1300014ba6a66c3431c05cc86c]). Expected version 1 got 0
I20251028 03:00:01.451149 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_29d2d7fc0d094968b83e38195008a440 [id=c9c59047d50a4419a9f16431bce05cd1] alter complete (version 2)
I20251028 03:00:01.451834 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 5155389485d54c1b82c45eb65364620f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.451486 8551 ts_tablet_manager.cc:1929] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.452775 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 0fcabc4a2fd44043be5451046ff5bd97 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.452982 9639 raft_consensus.cc:493] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.453061 9639 raft_consensus.cc:515] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.453416 9639 leader_election.cc:290] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:01.454077 8354 leader_election.cc:343] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 07d3e6c640434652a010942402c54716
I20251028 03:00:01.454474 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07d3e6c640434652a010942402c54716" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:01.454586 8273 raft_consensus.cc:2468] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:01.454779 9958 consensus_queue.cc:1048] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:01.455127 8356 leader_election.cc:304] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.455359 9639 raft_consensus.cc:2804] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:01.455425 9639 raft_consensus.cc:493] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:01.455464 9639 raft_consensus.cc:3060] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.451424 8552 ts_tablet_manager.cc:1929] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.456002 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3ea2d5c9d6d640ab90388e8696ebc2a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.455997 8551 log.cc:1199] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/b9b5fb938357499b8888ba29d23612bd
I20251028 03:00:01.456112 8552 log.cc:1199] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/b9b5fb938357499b8888ba29d23612bd
W20251028 03:00:01.456410 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ea2d5c9d6d640ab90388e8696ebc2a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.456521 9639 raft_consensus.cc:515] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.455101 8550 ts_tablet_manager.cc:1929] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.456950 9639 leader_election.cc:290] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:01.458148 8354 leader_election.cc:343] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 07d3e6c640434652a010942402c54716
I20251028 03:00:01.458509 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "07d3e6c640434652a010942402c54716" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:01.458612 8273 raft_consensus.cc:3060] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.457058 8550 log.cc:1199] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/b9b5fb938357499b8888ba29d23612bd
I20251028 03:00:01.459751 8273 raft_consensus.cc:2468] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:01.459872 8552 ts_tablet_manager.cc:1950] T b9b5fb938357499b8888ba29d23612bd P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.459751 8551 ts_tablet_manager.cc:1950] T b9b5fb938357499b8888ba29d23612bd P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.460695 8356 leader_election.cc:304] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.460842 8550 ts_tablet_manager.cc:1950] T b9b5fb938357499b8888ba29d23612bd P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.461885 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_08d693721d15425a85eb515a90b30096" } modify_external_catalogs: true
I20251028 03:00:01.461997 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_08d693721d15425a85eb515a90b30096" } modify_external_catalogs: true
I20251028 03:00:01.463497 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet b9b5fb938357499b8888ba29d23612bd (table default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]) successfully deleted
I20251028 03:00:01.464401 9639 raft_consensus.cc:2804] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:01.464489 9639 raft_consensus.cc:697] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:01.464362 8550 tablet_replica.cc:333] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.464618 9639 consensus_queue.cc:237] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.464924 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet b9b5fb938357499b8888ba29d23612bd (table default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]) successfully deleted
I20251028 03:00:01.465437 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet b9b5fb938357499b8888ba29d23612bd (table default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]) successfully deleted
I20251028 03:00:01.465884 8550 raft_consensus.cc:2243] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.466039 8551 tablet_replica.cc:333] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.466230 8552 tablet_replica.cc:333] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.467327 8551 raft_consensus.cc:2243] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.468225 8552 raft_consensus.cc:2243] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.467907 8550 raft_consensus.cc:2272] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.468425 8551 raft_consensus.cc:2272] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.469435 8552 raft_consensus.cc:2272] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.471659 8551 ts_tablet_manager.cc:1916] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.471552 8550 ts_tablet_manager.cc:1916] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.473284 8552 ts_tablet_manager.cc:1916] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.474270 7967 catalog_manager.cc:5649] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 03:00:01.474504 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet c0fbcdef1007456997db9fcf610f1e00 (table default.Table_8a0dcd295acc422e855b4fc5403cb7e3 [id=6e7f7ac227db47c98e3a115dddc9ceff]). Expected version 1 got 0
I20251028 03:00:01.474676 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 69e0358b593f4b55b6b4ec859e2c021d (table default.Table_49f2ad3b1c934eaeab556927604ddf1b [id=e88ffc49924340aa92a3aedd631330bf]). Expected version 1 got 0
I20251028 03:00:01.474870 7967 catalog_manager.cc:5649] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:01.479509 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 71cbcc2db2d4439fbda6f3ebc5f809e9
I20251028 03:00:01.480425 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.480616 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.480777 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.483582 8195 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:01.486912 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 585cb43a1daf4e90b45a1372a3f23450 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.491389 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet c0fbcdef1007456997db9fcf610f1e00 (table default.Table_8a0dcd295acc422e855b4fc5403cb7e3 [id=6e7f7ac227db47c98e3a115dddc9ceff]). Expected version 1 got 0
I20251028 03:00:01.491926 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 69e0358b593f4b55b6b4ec859e2c021d (table default.Table_49f2ad3b1c934eaeab556927604ddf1b [id=e88ffc49924340aa92a3aedd631330bf]). Expected version 1 got 0
I20251028 03:00:01.492146 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 00550a40f04046d19e9112814372da6c (table default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]). Expected version 1 got 0
I20251028 03:00:01.493785 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 585cb43a1daf4e90b45a1372a3f23450
I20251028 03:00:01.494169 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 585cb43a1daf4e90b45a1372a3f23450 deleted and replaced by tablet 72cdf48bee6649c5a3144876265dccfe
I20251028 03:00:01.494419 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 585cb43a1daf4e90b45a1372a3f23450 with delete_type TABLET_DATA_DELETED (replaced by tablet 72cdf48bee6649c5a3144876265dccfe) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.494583 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 585cb43a1daf4e90b45a1372a3f23450 with delete_type TABLET_DATA_DELETED (replaced by tablet 72cdf48bee6649c5a3144876265dccfe) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.494731 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 585cb43a1daf4e90b45a1372a3f23450 with delete_type TABLET_DATA_DELETED (replaced by tablet 72cdf48bee6649c5a3144876265dccfe) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.496891 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 07d3e6c640434652a010942402c54716 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e0e038d33bf45f693da251cd560ae2a) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.498127 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_aa11196c63eb4ee4ba2c3597325d460a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.498283 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 07d3e6c640434652a010942402c54716 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e0e038d33bf45f693da251cd560ae2a) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:01.498538 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa11196c63eb4ee4ba2c3597325d460a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.499766 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 0fcabc4a2fd44043be5451046ff5bd97 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.504475 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 00550a40f04046d19e9112814372da6c (table default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]). Expected version 1 got 0
I20251028 03:00:01.505270 8551 ts_tablet_manager.cc:1929] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.505314 8550 ts_tablet_manager.cc:1929] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.505779 8550 log.cc:1199] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9409f3ee379949809f823dfb7f73643b
I20251028 03:00:01.505820 8552 ts_tablet_manager.cc:1929] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.505904 8551 log.cc:1199] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9409f3ee379949809f823dfb7f73643b
I20251028 03:00:01.506351 8552 log.cc:1199] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9409f3ee379949809f823dfb7f73643b
I20251028 03:00:01.507386 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_3a3b5d6e921341349a2e4c02f4856164 [id=c4aadbebe43f451ea22467feaa6dafbb] alter complete (version 2)
I20251028 03:00:01.508950 8263 tablet_service.cc:1505] Processing CreateTablet for tablet e3c00803d8524f9396d505c2682bcc17 (DEFAULT_TABLE table=default.Table_80e34b41b16242e6968529b0ae18a3bf [id=07672f072b1e46d8ab09491ae77f9c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.509244 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3c00803d8524f9396d505c2682bcc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.509277 8386 tablet_service.cc:1505] Processing CreateTablet for tablet e3c00803d8524f9396d505c2682bcc17 (DEFAULT_TABLE table=default.Table_80e34b41b16242e6968529b0ae18a3bf [id=07672f072b1e46d8ab09491ae77f9c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.509519 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3c00803d8524f9396d505c2682bcc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.511611 8258 tablet_service.cc:1505] Processing CreateTablet for tablet a134446c707f4cf9abcf8ebfb923824d (DEFAULT_TABLE table=default.Table_326468a8b8bb4cab93e3f3c6bb66843b [id=4fc368aeac744f9ba9202e2c68b4a5c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.511853 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a134446c707f4cf9abcf8ebfb923824d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.511922 8118 tablet_service.cc:1505] Processing CreateTablet for tablet e3c00803d8524f9396d505c2682bcc17 (DEFAULT_TABLE table=default.Table_80e34b41b16242e6968529b0ae18a3bf [id=07672f072b1e46d8ab09491ae77f9c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.512337 8396 tablet_service.cc:1505] Processing CreateTablet for tablet a134446c707f4cf9abcf8ebfb923824d (DEFAULT_TABLE table=default.Table_326468a8b8bb4cab93e3f3c6bb66843b [id=4fc368aeac744f9ba9202e2c68b4a5c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.512550 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a134446c707f4cf9abcf8ebfb923824d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.514346 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 165158808fce41e6aa8f0c8302143e50 (DEFAULT_TABLE table=default.Table_13592b2f4e954570b468b492db58d8e1 [id=56de3340b70e4d98896c072d415d7517]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.514828 8117 tablet_service.cc:1505] Processing CreateTablet for tablet a134446c707f4cf9abcf8ebfb923824d (DEFAULT_TABLE table=default.Table_326468a8b8bb4cab93e3f3c6bb66843b [id=4fc368aeac744f9ba9202e2c68b4a5c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.515271 8390 tablet_service.cc:1505] Processing CreateTablet for tablet 165158808fce41e6aa8f0c8302143e50 (DEFAULT_TABLE table=default.Table_13592b2f4e954570b468b492db58d8e1 [id=56de3340b70e4d98896c072d415d7517]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.515516 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 165158808fce41e6aa8f0c8302143e50. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.516222 9913 tablet.cc:1722] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:01.518508 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3c00803d8524f9396d505c2682bcc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.518724 9903 mvcc.cc:204] Tried to move back new op lower bound from 7215597163206762496 to 7215597161392558080. Current Snapshot: MvccSnapshot[applied={T|T < 7215597163206762496}]
I20251028 03:00:01.518851 9906 tablet.cc:1722] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:01.520469 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 00550a40f04046d19e9112814372da6c (table default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]). Expected version 1 got 0
I20251028 03:00:01.520857 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a134446c707f4cf9abcf8ebfb923824d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.522493 8546 tablet_bootstrap.cc:492] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.522589 8546 ts_tablet_manager.cc:1403] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.189s user 0.002s sys 0.000s
I20251028 03:00:01.523114 8546 raft_consensus.cc:359] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.523236 8546 raft_consensus.cc:385] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.523265 8546 raft_consensus.cc:740] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.523404 8546 consensus_queue.cc:260] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.523699 8546 ts_tablet_manager.cc:1434] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.524309 8550 ts_tablet_manager.cc:1950] T 9409f3ee379949809f823dfb7f73643b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.524654 8552 ts_tablet_manager.cc:1950] T 9409f3ee379949809f823dfb7f73643b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.524500 8551 ts_tablet_manager.cc:1950] T 9409f3ee379949809f823dfb7f73643b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.528120 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9409f3ee379949809f823dfb7f73643b (table default.Table_0a1e2cd2aace496790ca53a8bdfbea25 [id=cf6c49c2909c46ce97c6f51a2e7f5300]) successfully deleted
I20251028 03:00:01.528832 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9409f3ee379949809f823dfb7f73643b (table default.Table_0a1e2cd2aace496790ca53a8bdfbea25 [id=cf6c49c2909c46ce97c6f51a2e7f5300]) successfully deleted
W20251028 03:00:01.528991 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 69c9f986f7004ee6a0b6bba624ae1f8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69c9f986f7004ee6a0b6bba624ae1f8d already in progress: creating tablet
W20251028 03:00:01.529102 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 already in progress: creating tablet
W20251028 03:00:01.529217 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 585cb43a1daf4e90b45a1372a3f23450 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 585cb43a1daf4e90b45a1372a3f23450 already in progress: creating tablet
W20251028 03:00:01.529382 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 69c9f986f7004ee6a0b6bba624ae1f8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69c9f986f7004ee6a0b6bba624ae1f8d already in progress: creating tablet
I20251028 03:00:01.529975 8551 tablet_replica.cc:333] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.530597 8551 raft_consensus.cc:2243] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.531209 8551 raft_consensus.cc:2272] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.532747 8551 ts_tablet_manager.cc:1916] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.535657 7967 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_e95d99935d7d4d94a16f5a826847ce32 [id=71026b1300014ba6a66c3431c05cc86c] alter complete (version 1)
I20251028 03:00:01.542061 8551 ts_tablet_manager.cc:1929] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.542425 8551 log.cc:1199] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/5155389485d54c1b82c45eb65364620f
I20251028 03:00:01.543351 8551 ts_tablet_manager.cc:1950] T 5155389485d54c1b82c45eb65364620f P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.545521 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 5155389485d54c1b82c45eb65364620f (table default.Table_b359d8a7e6724c21a2f0de83ddd841b7 [id=eb0e103661fc4bc7b27bb78fc333fd56]) successfully deleted
I20251028 03:00:01.546396 9909 tablet.cc:1722] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251028 03:00:01.546521 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 already in progress: creating tablet
W20251028 03:00:01.547379 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 585cb43a1daf4e90b45a1372a3f23450 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 585cb43a1daf4e90b45a1372a3f23450 already in progress: creating tablet
I20251028 03:00:01.547876 8551 tablet_replica.cc:333] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.548332 8551 raft_consensus.cc:2243] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.548907 8551 pending_rounds.cc:70] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 03:00:01.549101 8551 pending_rounds.cc:77] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597164411138048 op_type: NO_OP noop_request { }
I20251028 03:00:01.549404 8551 raft_consensus.cc:2889] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:01.549633 8551 raft_consensus.cc:2272] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.555404 8551 ts_tablet_manager.cc:1916] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.558202 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 00550a40f04046d19e9112814372da6c (table default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]). Expected version 1 got 0
I20251028 03:00:01.558249 8551 ts_tablet_manager.cc:1929] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.558318 8551 log.cc:1199] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/07d3e6c640434652a010942402c54716
I20251028 03:00:01.558585 8551 ts_tablet_manager.cc:1950] T 07d3e6c640434652a010942402c54716 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.559193 8551 tablet_replica.cc:333] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.559357 8551 raft_consensus.cc:2243] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.559460 8551 raft_consensus.cc:2272] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.560004 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 07d3e6c640434652a010942402c54716 (table default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]) successfully deleted
I20251028 03:00:01.560750 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b760e30f659c4e21b76a578756da6071"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.561309 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b760e30f659c4e21b76a578756da6071 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.564129 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 1c429d74609043818c89652f00a38a56 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.565218 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4d6d263beb42476ab582fad092f43a53" } modify_external_catalogs: true
I20251028 03:00:01.565389 8551 ts_tablet_manager.cc:1916] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.565413 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4d6d263beb42476ab582fad092f43a53" } modify_external_catalogs: true
I20251028 03:00:01.566860 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3fb2a054809d4428abf6fd4cee9a79cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.567759 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3fb2a054809d4428abf6fd4cee9a79cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.568598 8551 ts_tablet_manager.cc:1929] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.568732 8551 log.cc:1199] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/0fcabc4a2fd44043be5451046ff5bd97
I20251028 03:00:01.568747 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_03e0732cf61746cb89213075b54e35cc" } new_table_name: "default.Table_dbaf1e8395cd43a688b9952597cb6e8c" modify_external_catalogs: true
I20251028 03:00:01.569103 8551 ts_tablet_manager.cc:1950] T 0fcabc4a2fd44043be5451046ff5bd97 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.569919 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 0fcabc4a2fd44043be5451046ff5bd97 (table default.Table_31e973dfaa9b49f2bed3e9dfd11f7ae9 [id=f6bf7ec7e30f4c469ca1adde25fee2d2]) successfully deleted
I20251028 03:00:01.569962 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5d3da0f4d8af4dffb109a35707fe3a8f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.570302 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d3da0f4d8af4dffb109a35707fe3a8f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.572302 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 7a123640d3494675b58e878b6d4a77a6
I20251028 03:00:01.574565 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 7a123640d3494675b58e878b6d4a77a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.574892 7967 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 00550a40f04046d19e9112814372da6c (table default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5]). Expected version 1 got 0
W20251028 03:00:01.575078 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 7a123640d3494675b58e878b6d4a77a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a123640d3494675b58e878b6d4a77a6 already in progress: creating tablet
I20251028 03:00:01.575556 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet 7a123640d3494675b58e878b6d4a77a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.576133 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 7a123640d3494675b58e878b6d4a77a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.576316 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_122813ac12be41ba91423e5309fa8eab" } new_table_name: "default.Table_a2e6ca8fa3354866af0f7f6133b3e5ea" modify_external_catalogs: true
W20251028 03:00:01.576423 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 7a123640d3494675b58e878b6d4a77a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a123640d3494675b58e878b6d4a77a6 already in progress: creating tablet
I20251028 03:00:01.577458 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f1e09c9160e64a2bad978b4ebabb2c8f" } new_table_name: "default.Table_b5ab663b3ef34857bef3e7fc7ec5203e" modify_external_catalogs: true
I20251028 03:00:01.580569 8416 raft_consensus.cc:3060] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:01.580924 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a7fa9f23989d4bdba6545d0e681c78e5 consensus state has no leader
I20251028 03:00:01.581516 8142 raft_consensus.cc:1275] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:01.581698 8416 raft_consensus.cc:1275] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:01.581975 9958 consensus_queue.cc:1048] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:01.582094 9595 consensus_queue.cc:1048] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:01.592154 9913 tablet.cc:1722] T c0fbcdef1007456997db9fcf610f1e00 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:01.593822 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 165158808fce41e6aa8f0c8302143e50 (DEFAULT_TABLE table=default.Table_13592b2f4e954570b468b492db58d8e1 [id=56de3340b70e4d98896c072d415d7517]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.594206 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 165158808fce41e6aa8f0c8302143e50. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.594794 9909 tablet.cc:1722] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:01.595357 9906 tablet.cc:1722] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:01.595724 9913 tablet.cc:1722] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:01.599073 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 165158808fce41e6aa8f0c8302143e50. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.599695 9909 tablet.cc:1722] T c0fbcdef1007456997db9fcf610f1e00 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:01.599678 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet c0fbcdef1007456997db9fcf610f1e00 (table default.Table_8a0dcd295acc422e855b4fc5403cb7e3 [id=6e7f7ac227db47c98e3a115dddc9ceff]). Expected version 1 got 0
I20251028 03:00:01.601727 8546 tablet_bootstrap.cc:492] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:01.601877 9884 raft_consensus.cc:493] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.601958 9884 raft_consensus.cc:515] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.602308 9884 leader_election.cc:290] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.602705 8546 tablet_bootstrap.cc:654] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.602990 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ab2fe9132441c1834aea22561f02e9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:01.603116 8142 raft_consensus.cc:2468] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.603507 8552 tablet_replica.cc:333] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.603654 8552 raft_consensus.cc:2243] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.603708 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9409f3ee379949809f823dfb7f73643b (table default.Table_0a1e2cd2aace496790ca53a8bdfbea25 [id=cf6c49c2909c46ce97c6f51a2e7f5300]) successfully deleted
I20251028 03:00:01.603788 8552 pending_rounds.cc:70] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 03:00:01.603814 8552 pending_rounds.cc:77] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597164314238976 op_type: NO_OP noop_request { }
I20251028 03:00:01.603863 8552 raft_consensus.cc:2889] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:01.603920 8552 raft_consensus.cc:2272] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:01.603914 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 69c9f986f7004ee6a0b6bba624ae1f8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69c9f986f7004ee6a0b6bba624ae1f8d already in progress: creating tablet
I20251028 03:00:01.604064 8218 leader_election.cc:304] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:01.604457 9595 raft_consensus.cc:2804] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:01.604513 9595 raft_consensus.cc:493] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:01.604554 9595 raft_consensus.cc:3060] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.604883 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2847dd26debb407b821b676dc3df8c4a" } modify_external_catalogs: true
I20251028 03:00:01.604997 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2847dd26debb407b821b676dc3df8c4a" } modify_external_catalogs: true
I20251028 03:00:01.605583 9595 raft_consensus.cc:515] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.605907 9595 leader_election.cc:290] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.607751 8552 ts_tablet_manager.cc:1916] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.608870 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 16123c06b3d94acfa11145963bd31b6b (DEFAULT_TABLE table=default.Table_cd1078befff84198beaa6679f94a351c [id=fbb25f221b6f4123b1eb0274454f5ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.609160 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16123c06b3d94acfa11145963bd31b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.609454 8390 tablet_service.cc:1505] Processing CreateTablet for tablet 16123c06b3d94acfa11145963bd31b6b (DEFAULT_TABLE table=default.Table_cd1078befff84198beaa6679f94a351c [id=fbb25f221b6f4123b1eb0274454f5ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.609748 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16123c06b3d94acfa11145963bd31b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.611212 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ab2fe9132441c1834aea22561f02e9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:01.611351 8142 raft_consensus.cc:3060] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.611765 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 16123c06b3d94acfa11145963bd31b6b (DEFAULT_TABLE table=default.Table_cd1078befff84198beaa6679f94a351c [id=fbb25f221b6f4123b1eb0274454f5ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.611994 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16123c06b3d94acfa11145963bd31b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.612370 8142 raft_consensus.cc:2468] T 68ab2fe9132441c1834aea22561f02e9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:01.612670 8218 leader_election.cc:304] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:01.612865 9884 raft_consensus.cc:2804] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:01.612929 9884 raft_consensus.cc:697] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:01.613044 9884 consensus_queue.cc:237] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.613492 8396 tablet_service.cc:1505] Processing CreateTablet for tablet da46096a2001470ba1bd1c9660e4b0c5 (DEFAULT_TABLE table=default.Table_932eaf48b5614bbfb5d581cfd42f16d2 [id=206ed1f28426441c90d33a8cd0cb755b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.614107 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_fcc46f9512d14f0ea2eb69c442c50d69" } new_table_name: "default.Table_c11a363c1ffb42e6bca778a2776ec1fd" modify_external_catalogs: true
I20251028 03:00:01.615254 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ab2fe9132441c1834aea22561f02e9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.615655 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "68ab2fe9132441c1834aea22561f02e9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:01.615955 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_9a9b719b65a54f52af68c68c87bb18a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.616325 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_efe45b07a49349e7a7995ee1b0c4993e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.616349 9906 tablet.cc:1722] T c0fbcdef1007456997db9fcf610f1e00 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:01.616588 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 3a04270a2fbd49c294c4a2f24672036d
W20251028 03:00:01.616648 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efe45b07a49349e7a7995ee1b0c4993e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:01.617044 8220 leader_election.cc:343] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:01.617331 8220 leader_election.cc:343] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.617390 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 3a04270a2fbd49c294c4a2f24672036d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.617827 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c8a55dcfb9e94100866bac84b2ce9ecd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.618135 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8a55dcfb9e94100866bac84b2ce9ecd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.617852 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c149f5a7a9164664a7a699ee42b93697" } modify_external_catalogs: true
I20251028 03:00:01.619565 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c149f5a7a9164664a7a699ee42b93697" } modify_external_catalogs: true
I20251028 03:00:01.620491 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da46096a2001470ba1bd1c9660e4b0c5. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:01.620738 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 3a04270a2fbd49c294c4a2f24672036d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a04270a2fbd49c294c4a2f24672036d already in progress: creating tablet
I20251028 03:00:01.621094 8552 ts_tablet_manager.cc:1929] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.621156 8552 log.cc:1199] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/0fcabc4a2fd44043be5451046ff5bd97
I20251028 03:00:01.621399 8552 ts_tablet_manager.cc:1950] T 0fcabc4a2fd44043be5451046ff5bd97 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.622081 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 0fcabc4a2fd44043be5451046ff5bd97 (table default.Table_31e973dfaa9b49f2bed3e9dfd11f7ae9 [id=f6bf7ec7e30f4c469ca1adde25fee2d2]) successfully deleted
W20251028 03:00:01.624050 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71cbcc2db2d4439fbda6f3ebc5f809e9 already in progress: creating tablet
I20251028 03:00:01.627180 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet bedb4fd3b46f4453b194cfaea0c3c701
I20251028 03:00:01.627882 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet bedb4fd3b46f4453b194cfaea0c3c701 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.628824 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet bedb4fd3b46f4453b194cfaea0c3c701 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:01.628859 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet bedb4fd3b46f4453b194cfaea0c3c701 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bedb4fd3b46f4453b194cfaea0c3c701 already in progress: creating tablet
I20251028 03:00:01.629076 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet bedb4fd3b46f4453b194cfaea0c3c701 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:01.629287 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet bedb4fd3b46f4453b194cfaea0c3c701 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bedb4fd3b46f4453b194cfaea0c3c701 already in progress: creating tablet
I20251028 03:00:01.629580 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 3a04270a2fbd49c294c4a2f24672036d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.629832 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_db269528fac84afd92d7be143be14944" } new_table_name: "default.Table_2b53e6b3d54c45ad8e42a41bc9da9751" modify_external_catalogs: true
I20251028 03:00:01.633486 7965 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_8a0dcd295acc422e855b4fc5403cb7e3 [id=6e7f7ac227db47c98e3a115dddc9ceff] alter complete (version 1)
I20251028 03:00:01.635391 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 3a04270a2fbd49c294c4a2f24672036d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:01.635949 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 3a04270a2fbd49c294c4a2f24672036d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a04270a2fbd49c294c4a2f24672036d already in progress: creating tablet
W20251028 03:00:01.636374 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a9b719b65a54f52af68c68c87bb18a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.638618 9840 consensus_queue.cc:1048] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:01.639047 9909 tablet.cc:1722] T e71f5d12fe2e4822b14c72c2b880c623 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:01.640257 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d3251b07042c44ca9807ffa0df1274fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.640736 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3251b07042c44ca9807ffa0df1274fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.642143 8544 tablet_bootstrap.cc:492] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:01.642238 8544 ts_tablet_manager.cc:1403] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.475s user 0.000s sys 0.003s
I20251028 03:00:01.642716 8544 raft_consensus.cc:359] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.642839 8544 raft_consensus.cc:385] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.642874 8544 raft_consensus.cc:740] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.642887 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1c980a2eaf3c4a458846ce8e022c35d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.642997 8544 consensus_queue.cc:260] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:01.643253 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c980a2eaf3c4a458846ce8e022c35d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.643328 8544 ts_tablet_manager.cc:1434] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:01.643520 8544 tablet_bootstrap.cc:492] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:01.644522 8544 tablet_bootstrap.cc:654] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:01.645254 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eff471c288544f0fbe4029bb3c52b929 consensus state has no leader
I20251028 03:00:01.645318 9913 tablet.cc:1722] T e71f5d12fe2e4822b14c72c2b880c623 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:01.645709 10192 tablet.cc:1722] T e71f5d12fe2e4822b14c72c2b880c623 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:01.646342 8552 tablet_replica.cc:333] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.646899 8552 raft_consensus.cc:2243] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.647365 8552 raft_consensus.cc:2272] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:01.648015 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 585cb43a1daf4e90b45a1372a3f23450 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 585cb43a1daf4e90b45a1372a3f23450 already in progress: creating tablet
I20251028 03:00:01.648798 7965 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_a2e6ca8fa3354866af0f7f6133b3e5ea [id=3131dc70f17c4cdf84d08ab26c718e78] alter complete (version 1)
I20251028 03:00:01.653426 9906 tablet.cc:1722] T 1979906599754f7ba049c7a9b23bf26d P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 03:00:01.653537 9909 tablet.cc:1722] T 1979906599754f7ba049c7a9b23bf26d P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 03:00:01.653623 8263 tablet_service.cc:1505] Processing CreateTablet for tablet da46096a2001470ba1bd1c9660e4b0c5 (DEFAULT_TABLE table=default.Table_932eaf48b5614bbfb5d581cfd42f16d2 [id=206ed1f28426441c90d33a8cd0cb755b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.653923 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da46096a2001470ba1bd1c9660e4b0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.658392 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 50e589efb98b4715a95e40b98512b86b (DEFAULT_TABLE table=default.Table_934fd45916ce4d068d19c1329ac0836b [id=743d97e640734617a4438f30d694b730]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.658685 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e589efb98b4715a95e40b98512b86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.658972 8390 tablet_service.cc:1505] Processing CreateTablet for tablet 50e589efb98b4715a95e40b98512b86b (DEFAULT_TABLE table=default.Table_934fd45916ce4d068d19c1329ac0836b [id=743d97e640734617a4438f30d694b730]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.660254 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e589efb98b4715a95e40b98512b86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.661010 8468 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:01.661566 8117 tablet_service.cc:1505] Processing CreateTablet for tablet da46096a2001470ba1bd1c9660e4b0c5 (DEFAULT_TABLE table=default.Table_932eaf48b5614bbfb5d581cfd42f16d2 [id=206ed1f28426441c90d33a8cd0cb755b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.661825 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da46096a2001470ba1bd1c9660e4b0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.664764 8552 ts_tablet_manager.cc:1916] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.664518 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d6b0104aa11a449198e0c05a7f13c6a4" } modify_external_catalogs: true
I20251028 03:00:01.664976 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d6b0104aa11a449198e0c05a7f13c6a4" } modify_external_catalogs: true
I20251028 03:00:01.665444 10112 raft_consensus.cc:493] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.665534 10112 raft_consensus.cc:515] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.665936 10112 leader_election.cc:290] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:01.666508 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9277a4ffd0ca4bf7afd29e1c6ab33bc9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:01.666889 8083 leader_election.cc:343] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.668939 8545 tablet_bootstrap.cc:492] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.669049 8545 ts_tablet_manager.cc:1403] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.309s user 0.002s sys 0.000s
I20251028 03:00:01.669478 8545 raft_consensus.cc:359] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.669589 8545 raft_consensus.cc:385] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.669623 8545 raft_consensus.cc:740] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.669772 8545 consensus_queue.cc:260] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.666550 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9277a4ffd0ca4bf7afd29e1c6ab33bc9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:01.670095 8545 ts_tablet_manager.cc:1434] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.670300 8545 tablet_bootstrap.cc:492] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 03:00:01.670375 8083 leader_election.cc:343] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.670446 8083 leader_election.cc:304] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:01.670588 10112 raft_consensus.cc:2749] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.667846 9913 tablet.cc:1722] T 1979906599754f7ba049c7a9b23bf26d P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 03:00:01.670990 8263 tablet_service.cc:1505] Processing CreateTablet for tablet a734e37a3b984067bc1aec42a52f8bbb (DEFAULT_TABLE table=default.Table_0c654fd0364e44fa8d063a3c40273316 [id=c40c63278a93445296664da64e98304f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.671262 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a734e37a3b984067bc1aec42a52f8bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.673503 8545 tablet_bootstrap.cc:654] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.674451 8552 ts_tablet_manager.cc:1929] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.674533 8552 log.cc:1199] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/07d3e6c640434652a010942402c54716
I20251028 03:00:01.674783 8552 ts_tablet_manager.cc:1950] T 07d3e6c640434652a010942402c54716 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.675701 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 07d3e6c640434652a010942402c54716 (table default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]) successfully deleted
I20251028 03:00:01.675920 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 50e589efb98b4715a95e40b98512b86b (DEFAULT_TABLE table=default.Table_934fd45916ce4d068d19c1329ac0836b [id=743d97e640734617a4438f30d694b730]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.676213 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e589efb98b4715a95e40b98512b86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.676368 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ec5ff388e162425e96cc5c12f2e8f804"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.676790 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec5ff388e162425e96cc5c12f2e8f804 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.678177 8390 tablet_service.cc:1505] Processing CreateTablet for tablet a734e37a3b984067bc1aec42a52f8bbb (DEFAULT_TABLE table=default.Table_0c654fd0364e44fa8d063a3c40273316 [id=c40c63278a93445296664da64e98304f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.679543 8546 tablet_bootstrap.cc:492] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.679639 8546 ts_tablet_manager.cc:1403] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.078s user 0.001s sys 0.000s
I20251028 03:00:01.680111 8546 raft_consensus.cc:359] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.680224 8546 raft_consensus.cc:385] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.680258 8546 raft_consensus.cc:740] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.680387 8546 consensus_queue.cc:260] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.680676 8546 ts_tablet_manager.cc:1434] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:01.680869 8546 tablet_bootstrap.cc:492] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:01.682914 8546 tablet_bootstrap.cc:654] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.684113 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e5fa1e8f3825450685059a8a45433ef8" } modify_external_catalogs: true
I20251028 03:00:01.684326 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e5fa1e8f3825450685059a8a45433ef8" } modify_external_catalogs: true
I20251028 03:00:01.684841 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a734e37a3b984067bc1aec42a52f8bbb. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:01.685317 10101 log.cc:927] Time spent T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: Append to log took a long time: real 0.121s user 0.001s sys 0.000s
I20251028 03:00:01.687700 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 81b521c9f57844bb9873f6b26b5cf661 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.688582 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet fd8b66680c6b4e64b06755270b065db4
I20251028 03:00:01.689440 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet fd8b66680c6b4e64b06755270b065db4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:01.690021 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet fd8b66680c6b4e64b06755270b065db4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fd8b66680c6b4e64b06755270b065db4 already in progress: creating tablet
I20251028 03:00:01.690593 8553 consensus_queue.cc:1048] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:01.691020 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet fd8b66680c6b4e64b06755270b065db4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.691401 8544 tablet_bootstrap.cc:492] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:01.691504 8544 ts_tablet_manager.cc:1403] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.000s
I20251028 03:00:01.691962 8544 raft_consensus.cc:359] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.692070 8544 raft_consensus.cc:385] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.692104 8544 raft_consensus.cc:740] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
W20251028 03:00:01.692116 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet fd8b66680c6b4e64b06755270b065db4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fd8b66680c6b4e64b06755270b065db4 already in progress: creating tablet
I20251028 03:00:01.692239 8544 consensus_queue.cc:260] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:01.692348 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 7a123640d3494675b58e878b6d4a77a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a123640d3494675b58e878b6d4a77a6 already in progress: creating tablet
I20251028 03:00:01.692543 8544 ts_tablet_manager.cc:1434] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 03:00:01.692531 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet bedb4fd3b46f4453b194cfaea0c3c701 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bedb4fd3b46f4453b194cfaea0c3c701 already in progress: creating tablet
I20251028 03:00:01.692731 8544 tablet_bootstrap.cc:492] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:01.693181 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 1de3acf65f744368ab31dc1c82ac554f
I20251028 03:00:01.693638 8544 tablet_bootstrap.cc:654] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.693733 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet 1de3acf65f744368ab31dc1c82ac554f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.693889 8551 tablet_replica.cc:333] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.693922 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 1de3acf65f744368ab31dc1c82ac554f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.694043 8551 raft_consensus.cc:2243] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
W20251028 03:00:01.694149 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 3a04270a2fbd49c294c4a2f24672036d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a04270a2fbd49c294c4a2f24672036d already in progress: creating tablet
I20251028 03:00:01.694180 8551 raft_consensus.cc:2272] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.694598 8551 ts_tablet_manager.cc:1916] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.697016 8551 ts_tablet_manager.cc:1929] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.697679 8551 log.cc:1199] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/1de3acf65f744368ab31dc1c82ac554f
I20251028 03:00:01.698033 8551 ts_tablet_manager.cc:1950] T 1de3acf65f744368ab31dc1c82ac554f P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.699158 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 1de3acf65f744368ab31dc1c82ac554f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.699431 8550 tablet_replica.cc:333] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.699586 8550 raft_consensus.cc:2243] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.699714 8550 raft_consensus.cc:2272] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.700107 8550 ts_tablet_manager.cc:1916] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.701228 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 1de3acf65f744368ab31dc1c82ac554f (table default.Table_e5fa1e8f3825450685059a8a45433ef8 [id=739f0673774d472bbb62f176b70998bc]) successfully deleted
I20251028 03:00:01.704844 8118 tablet_service.cc:1505] Processing CreateTablet for tablet f9951c153df74554aba611aa4be7c645 (DEFAULT_TABLE table=default.Table_f4eb60b8380645758ce2cadfaff1330a [id=e5dbece40326467088d7435702d8acbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.705600 8117 tablet_service.cc:1505] Processing CreateTablet for tablet a734e37a3b984067bc1aec42a52f8bbb (DEFAULT_TABLE table=default.Table_0c654fd0364e44fa8d063a3c40273316 [id=c40c63278a93445296664da64e98304f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.708374 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4d5a8f2489b24f4eb6173b10a9328e41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.709151 9190 consensus_queue.cc:1048] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 03:00:01.709260 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d5a8f2489b24f4eb6173b10a9328e41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.709965 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 401d810e96f34cff8da2cdd872dfa03b (DEFAULT_TABLE table=default.Table_620249416bb9473b90b7fb58a6032a1b [id=ed189d8bdc2c492482b13baffddf1da4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.710258 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 401d810e96f34cff8da2cdd872dfa03b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.710430 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 401d810e96f34cff8da2cdd872dfa03b (DEFAULT_TABLE table=default.Table_620249416bb9473b90b7fb58a6032a1b [id=ed189d8bdc2c492482b13baffddf1da4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.711290 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 401d810e96f34cff8da2cdd872dfa03b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.713538 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9951c153df74554aba611aa4be7c645. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.713730 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a734e37a3b984067bc1aec42a52f8bbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.713876 8550 ts_tablet_manager.cc:1929] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.713941 8550 log.cc:1199] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/1de3acf65f744368ab31dc1c82ac554f
I20251028 03:00:01.714636 8550 ts_tablet_manager.cc:1950] T 1de3acf65f744368ab31dc1c82ac554f P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.715214 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 1de3acf65f744368ab31dc1c82ac554f (table default.Table_e5fa1e8f3825450685059a8a45433ef8 [id=739f0673774d472bbb62f176b70998bc]) successfully deleted
I20251028 03:00:01.715585 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 81b521c9f57844bb9873f6b26b5cf661
I20251028 03:00:01.716012 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet ba40ba2c066e4c94885a595fd1c4445e (table default.Table_0bed08535f19450db1c432dd032850e5 [id=25d0c2f2faef473b94ce2621281c08b0]). Expected version 1 got 0
I20251028 03:00:01.716150 7966 catalog_manager.cc:6719] ReplaceTablet: tablet 81b521c9f57844bb9873f6b26b5cf661 deleted and replaced by tablet 9b8fe701ffd44cffa62ec55be9edfe31
I20251028 03:00:01.717108 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 81b521c9f57844bb9873f6b26b5cf661 with delete_type TABLET_DATA_DELETED (replaced by tablet 9b8fe701ffd44cffa62ec55be9edfe31) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.718429 8551 tablet_replica.cc:333] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.718588 8551 raft_consensus.cc:2243] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.718717 8551 raft_consensus.cc:2272] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.719143 8551 ts_tablet_manager.cc:1916] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.719823 8544 tablet_bootstrap.cc:492] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:01.719914 8544 ts_tablet_manager.cc:1403] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20251028 03:00:01.720312 8544 raft_consensus.cc:359] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.720422 8544 raft_consensus.cc:385] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.720454 8544 raft_consensus.cc:740] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.720575 8544 consensus_queue.cc:260] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.720683 7967 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet aa36c6870c1b4a459f0a41c84855a76e (table default.Table_d51065c1d57c4857bc5765d1baf9da06 [id=3815e9736bd34d668ec7f7e113e11e6f]). Expected version 1 got 0
I20251028 03:00:01.720909 8544 ts_tablet_manager.cc:1434] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.721088 8544 tablet_bootstrap.cc:492] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:01.722013 8551 ts_tablet_manager.cc:1929] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.722081 8551 log.cc:1199] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/81b521c9f57844bb9873f6b26b5cf661
I20251028 03:00:01.722242 8544 tablet_bootstrap.cc:654] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.722335 8551 ts_tablet_manager.cc:1950] T 81b521c9f57844bb9873f6b26b5cf661 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.723029 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 81b521c9f57844bb9873f6b26b5cf661 (table default.Table_c14c90440d424269b7a05b90f49e57e6 [id=5f82b9596e5842deb4d45e19c91629bb]) successfully deleted
I20251028 03:00:01.723915 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 347e824873f8483d974e85440c70c3d2 (DEFAULT_TABLE table=default.Table_78fb1bab55774710bbc8014a9085fb0d [id=43844eeb88004b968a1996a1e77ab94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.724157 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 347e824873f8483d974e85440c70c3d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.724402 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 (DEFAULT_TABLE table=default.Table_7cc757dea1fe4568957409286675204d [id=a778ccdafab64d45bc2041d21a2f9ddc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.724601 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.725497 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 81b521c9f57844bb9873f6b26b5cf661 with delete_type TABLET_DATA_DELETED (replaced by tablet 9b8fe701ffd44cffa62ec55be9edfe31) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.726151 8550 tablet_replica.cc:333] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.726321 8550 raft_consensus.cc:2243] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.726449 8550 raft_consensus.cc:2272] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.726518 8400 tablet_service.cc:1505] Processing CreateTablet for tablet d2595215c1264d8eb3cc56599fac1780 (DEFAULT_TABLE table=default.Table_7f9762941e0044468dc41a8d0974ba58 [id=78c397f4217c46849b1c2ba6aec5324d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.726827 8550 ts_tablet_manager.cc:1916] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.727512 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2595215c1264d8eb3cc56599fac1780. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.729984 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 347e824873f8483d974e85440c70c3d2 (DEFAULT_TABLE table=default.Table_78fb1bab55774710bbc8014a9085fb0d [id=43844eeb88004b968a1996a1e77ab94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.730316 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 347e824873f8483d974e85440c70c3d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.694326 8552 tablet_replica.cc:333] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.719449 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 347e824873f8483d974e85440c70c3d2 (DEFAULT_TABLE table=default.Table_78fb1bab55774710bbc8014a9085fb0d [id=43844eeb88004b968a1996a1e77ab94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.699092 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet fd8b66680c6b4e64b06755270b065db4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.727135 8251 tablet_service.cc:1505] Processing CreateTablet for tablet d2595215c1264d8eb3cc56599fac1780 (DEFAULT_TABLE table=default.Table_7f9762941e0044468dc41a8d0974ba58 [id=78c397f4217c46849b1c2ba6aec5324d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.707051 8260 tablet_service.cc:1505] Processing CreateTablet for tablet f9951c153df74554aba611aa4be7c645 (DEFAULT_TABLE table=default.Table_f4eb60b8380645758ce2cadfaff1330a [id=e5dbece40326467088d7435702d8acbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.711513 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 401d810e96f34cff8da2cdd872dfa03b (DEFAULT_TABLE table=default.Table_620249416bb9473b90b7fb58a6032a1b [id=ed189d8bdc2c492482b13baffddf1da4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.731940 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 (DEFAULT_TABLE table=default.Table_7cc757dea1fe4568957409286675204d [id=a778ccdafab64d45bc2041d21a2f9ddc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.699503 8553 consensus_queue.cc:1048] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20251028 03:00:01.732168 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.707145 9595 raft_consensus.cc:493] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.732367 9595 raft_consensus.cc:515] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.732736 9595 leader_election.cc:290] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.717664 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 81b521c9f57844bb9873f6b26b5cf661 with delete_type TABLET_DATA_DELETED (replaced by tablet 9b8fe701ffd44cffa62ec55be9edfe31) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.733368 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 387c4d8acc2e486f8da70110c7264923 (DEFAULT_TABLE table=default.Table_a932ef07b1b9433ca569bbae983a4743 [id=7a85ab3a9d2243658e01dcd1bc472194]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.733608 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 387c4d8acc2e486f8da70110c7264923. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.725426 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 (DEFAULT_TABLE table=default.Table_7cc757dea1fe4568957409286675204d [id=a778ccdafab64d45bc2041d21a2f9ddc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.734192 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "69e0358b593f4b55b6b4ec859e2c021d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.734301 8420 raft_consensus.cc:2468] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.735097 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 387c4d8acc2e486f8da70110c7264923 (DEFAULT_TABLE table=default.Table_a932ef07b1b9433ca569bbae983a4743 [id=7a85ab3a9d2243658e01dcd1bc472194]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.735500 8220 leader_election.cc:304] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:01.735652 9884 raft_consensus.cc:2804] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:01.735692 9884 raft_consensus.cc:493] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:01.735738 9884 raft_consensus.cc:3060] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.736195 8263 tablet_service.cc:1505] Processing CreateTablet for tablet c47a6bf438c042e1af940f1ae5a64093 (DEFAULT_TABLE table=default.Table_5b4d3ba3f03648b0b3613ccc0c1acc6d [id=85d94d335c1144f48ae805bbffe226c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.736763 8400 tablet_service.cc:1505] Processing CreateTablet for tablet c47a6bf438c042e1af940f1ae5a64093 (DEFAULT_TABLE table=default.Table_5b4d3ba3f03648b0b3613ccc0c1acc6d [id=85d94d335c1144f48ae805bbffe226c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.736748 9884 raft_consensus.cc:515] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.736953 8544 tablet_bootstrap.cc:492] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:01.737000 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47a6bf438c042e1af940f1ae5a64093. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.737041 8544 ts_tablet_manager.cc:1403] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.001s
I20251028 03:00:01.737115 9884 leader_election.cc:290] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.737562 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "69e0358b593f4b55b6b4ec859e2c021d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:01.737543 8544 raft_consensus.cc:359] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.737648 8544 raft_consensus.cc:385] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.737658 8420 raft_consensus.cc:3060] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.737679 8544 raft_consensus.cc:740] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.737824 8544 consensus_queue.cc:260] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.738198 8544 ts_tablet_manager.cc:1434] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.738361 8544 tablet_bootstrap.cc:492] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:01.738574 8420 raft_consensus.cc:2468] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:01.738885 8220 leader_election.cc:304] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:01.739249 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "69e0358b593f4b55b6b4ec859e2c021d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:01.739367 9884 raft_consensus.cc:2804] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:01.739395 8142 raft_consensus.cc:2468] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.739451 9884 raft_consensus.cc:697] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:01.739475 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "69e0358b593f4b55b6b4ec859e2c021d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:01.739563 8144 raft_consensus.cc:3060] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.739585 9884 consensus_queue.cc:237] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.739945 8544 tablet_bootstrap.cc:654] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.740554 8144 raft_consensus.cc:2468] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:01.731678 8552 raft_consensus.cc:2243] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.741098 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 347e824873f8483d974e85440c70c3d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.741101 8122 tablet_service.cc:1505] Processing CreateTablet for tablet d2595215c1264d8eb3cc56599fac1780 (DEFAULT_TABLE table=default.Table_7f9762941e0044468dc41a8d0974ba58 [id=78c397f4217c46849b1c2ba6aec5324d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.741174 8552 raft_consensus.cc:2272] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.741340 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2595215c1264d8eb3cc56599fac1780. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.741571 8552 ts_tablet_manager.cc:1916] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.742903 8390 tablet_service.cc:1505] Processing CreateTablet for tablet f9951c153df74554aba611aa4be7c645 (DEFAULT_TABLE table=default.Table_f4eb60b8380645758ce2cadfaff1330a [id=e5dbece40326467088d7435702d8acbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.743194 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9951c153df74554aba611aa4be7c645. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.743427 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9951c153df74554aba611aa4be7c645. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.743820 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2595215c1264d8eb3cc56599fac1780. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.744035 7965 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_c11a363c1ffb42e6bca778a2776ec1fd [id=9fcc70dd88724a28b404e9d4109015aa] alter complete (version 3)
I20251028 03:00:01.745867 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 387c4d8acc2e486f8da70110c7264923. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.746186 7966 catalog_manager.cc:5649] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 03:00:01.746503 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet aa36c6870c1b4a459f0a41c84855a76e (table default.Table_d51065c1d57c4857bc5765d1baf9da06 [id=3815e9736bd34d668ec7f7e113e11e6f]). Expected version 1 got 0
I20251028 03:00:01.746601 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 2bf100407b034626b4a9f7e2340f9cf0 (table default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262]). Expected version 2 got 0
I20251028 03:00:01.748883 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.749219 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 401d810e96f34cff8da2cdd872dfa03b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.730316 8550 ts_tablet_manager.cc:1929] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.754103 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet aa36c6870c1b4a459f0a41c84855a76e (table default.Table_d51065c1d57c4857bc5765d1baf9da06 [id=3815e9736bd34d668ec7f7e113e11e6f]). Expected version 1 got 0
I20251028 03:00:01.754801 8545 tablet_bootstrap.cc:492] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.754892 8545 ts_tablet_manager.cc:1403] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.085s user 0.002s sys 0.000s
I20251028 03:00:01.755178 8550 log.cc:1199] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/81b521c9f57844bb9873f6b26b5cf661
I20251028 03:00:01.755385 8545 raft_consensus.cc:359] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.755489 8545 raft_consensus.cc:385] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.755519 8545 raft_consensus.cc:740] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.755631 8545 consensus_queue.cc:260] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.755661 8550 ts_tablet_manager.cc:1950] T 81b521c9f57844bb9873f6b26b5cf661 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.755882 8545 ts_tablet_manager.cc:1434] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.756067 8545 tablet_bootstrap.cc:492] T 68ab2fe9132441c1834aea22561f02e9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.759282 8545 tablet_bootstrap.cc:654] T 68ab2fe9132441c1834aea22561f02e9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.762988 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47a6bf438c042e1af940f1ae5a64093. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.762215 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet a7fa9f23989d4bdba6545d0e681c78e5 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.765209 8546 tablet_bootstrap.cc:492] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.765290 8546 ts_tablet_manager.cc:1403] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.084s user 0.002s sys 0.001s
I20251028 03:00:01.765764 8546 raft_consensus.cc:359] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.765877 8546 raft_consensus.cc:385] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.765908 8546 raft_consensus.cc:740] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.766021 8546 consensus_queue.cc:260] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.766294 8546 ts_tablet_manager.cc:1434] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.766467 8546 tablet_bootstrap.cc:492] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:01.768291 7967 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_b5ab663b3ef34857bef3e7fc7ec5203e [id=f1872634680f477e9f44007a30ef02bd] alter complete (version 2)
I20251028 03:00:01.769189 8546 tablet_bootstrap.cc:654] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.770766 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 81b521c9f57844bb9873f6b26b5cf661 (table default.Table_c14c90440d424269b7a05b90f49e57e6 [id=5f82b9596e5842deb4d45e19c91629bb]) successfully deleted
I20251028 03:00:01.771529 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 69e0358b593f4b55b6b4ec859e2c021d from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.773200 8552 ts_tablet_manager.cc:1929] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.773339 8552 log.cc:1199] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/1de3acf65f744368ab31dc1c82ac554f
I20251028 03:00:01.773677 8552 ts_tablet_manager.cc:1950] T 1de3acf65f744368ab31dc1c82ac554f P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.774624 8552 tablet_replica.cc:333] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.774825 8552 raft_consensus.cc:2243] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.775068 8552 raft_consensus.cc:2272] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.775388 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 1de3acf65f744368ab31dc1c82ac554f (table default.Table_e5fa1e8f3825450685059a8a45433ef8 [id=739f0673774d472bbb62f176b70998bc]) successfully deleted
I20251028 03:00:01.775517 8552 ts_tablet_manager.cc:1916] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:01.775560 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet fd8b66680c6b4e64b06755270b065db4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fd8b66680c6b4e64b06755270b065db4 already in progress: creating tablet
I20251028 03:00:01.775974 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d0b0de6991ea465f9b83903824f98ad9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.776407 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.777943 8552 ts_tablet_manager.cc:1929] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.778007 8552 log.cc:1199] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/81b521c9f57844bb9873f6b26b5cf661
I20251028 03:00:01.778199 8550 tablet_replica.cc:333] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.778275 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 9621e42e6a324ecda50fe83c0c53119c with delete_type TABLET_DATA_DELETED (replaced by tablet bedb4fd3b46f4453b194cfaea0c3c701) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.778349 8550 raft_consensus.cc:2243] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.778452 8550 raft_consensus.cc:2272] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.778555 8122 tablet_service.cc:1505] Processing CreateTablet for tablet c47a6bf438c042e1af940f1ae5a64093 (DEFAULT_TABLE table=default.Table_5b4d3ba3f03648b0b3613ccc0c1acc6d [id=85d94d335c1144f48ae805bbffe226c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.778805 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c47a6bf438c042e1af940f1ae5a64093. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:01.779255 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0b0de6991ea465f9b83903824f98ad9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.779680 8550 ts_tablet_manager.cc:1916] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.781697 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e95d99935d7d4d94a16f5a826847ce32" } modify_external_catalogs: true
I20251028 03:00:01.781822 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e95d99935d7d4d94a16f5a826847ce32" } modify_external_catalogs: true
I20251028 03:00:01.782334 8550 ts_tablet_manager.cc:1929] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.782399 8550 log.cc:1199] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9277a4ffd0ca4bf7afd29e1c6ab33bc9
I20251028 03:00:01.782611 8550 ts_tablet_manager.cc:1950] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.782845 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6f8c1a5b100b4e0b8db85770d0a3108d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.782989 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 (table default.Table_eca57c87ce0240a0844cc210e4dbdd85 [id=289693853ee848e785d5460a21ae5063]) successfully deleted
W20251028 03:00:01.783207 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f8c1a5b100b4e0b8db85770d0a3108d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.784876 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_555cd9726353408bbafada707d406c8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.785082 8550 tablet_replica.cc:333] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.785221 8550 raft_consensus.cc:2243] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 03:00:01.785230 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_555cd9726353408bbafada707d406c8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.785331 8550 raft_consensus.cc:2272] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.785650 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 387c4d8acc2e486f8da70110c7264923 (DEFAULT_TABLE table=default.Table_a932ef07b1b9433ca569bbae983a4743 [id=7a85ab3a9d2243658e01dcd1bc472194]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.784003 8546 tablet_bootstrap.cc:492] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.785905 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 387c4d8acc2e486f8da70110c7264923. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.785912 8546 ts_tablet_manager.cc:1403] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20251028 03:00:01.786422 8546 raft_consensus.cc:359] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.786520 8546 raft_consensus.cc:385] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.786545 8546 raft_consensus.cc:740] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.786648 8546 consensus_queue.cc:260] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.784914 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0bed08535f19450db1c432dd032850e5" } new_table_name: "default.Table_b2cfdddecdf14efcb61d58a4646b792f" modify_external_catalogs: true
I20251028 03:00:01.786897 8546 ts_tablet_manager.cc:1434] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.787065 8546 tablet_bootstrap.cc:492] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:01.787896 8550 ts_tablet_manager.cc:1916] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.788067 8546 tablet_bootstrap.cc:654] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.790220 8546 tablet_bootstrap.cc:492] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:01.790421 8550 ts_tablet_manager.cc:1929] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.790477 8550 log.cc:1199] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9621e42e6a324ecda50fe83c0c53119c
I20251028 03:00:01.790665 8550 ts_tablet_manager.cc:1950] T 9621e42e6a324ecda50fe83c0c53119c P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.791059 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9621e42e6a324ecda50fe83c0c53119c (table default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]) successfully deleted
I20251028 03:00:01.795729 8544 tablet_bootstrap.cc:492] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:01.795833 8544 ts_tablet_manager.cc:1403] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.058s user 0.001s sys 0.000s
I20251028 03:00:01.796279 8544 raft_consensus.cc:359] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.796373 8544 raft_consensus.cc:385] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.796399 8544 raft_consensus.cc:740] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.796536 8544 consensus_queue.cc:260] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.798297 8544 ts_tablet_manager.cc:1434] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:01.798496 8544 tablet_bootstrap.cc:492] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:01.801452 8545 tablet_bootstrap.cc:492] T 68ab2fe9132441c1834aea22561f02e9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.801556 8545 ts_tablet_manager.cc:1403] T 68ab2fe9132441c1834aea22561f02e9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.046s user 0.001s sys 0.001s
I20251028 03:00:01.802069 8545 raft_consensus.cc:359] T 68ab2fe9132441c1834aea22561f02e9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.802179 8545 raft_consensus.cc:385] T 68ab2fe9132441c1834aea22561f02e9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.802217 8545 raft_consensus.cc:740] T 68ab2fe9132441c1834aea22561f02e9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.802330 8545 consensus_queue.cc:260] T 68ab2fe9132441c1834aea22561f02e9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.802507 8546 ts_tablet_manager.cc:1403] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 03:00:01.802644 8545 ts_tablet_manager.cc:1434] T 68ab2fe9132441c1834aea22561f02e9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:01.802850 8545 tablet_bootstrap.cc:492] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.803053 8546 raft_consensus.cc:359] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.803164 8546 raft_consensus.cc:385] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.803202 8546 raft_consensus.cc:740] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.803277 8544 tablet_bootstrap.cc:654] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.803359 8546 consensus_queue.cc:260] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.803680 8546 ts_tablet_manager.cc:1434] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:01.803862 8546 tablet_bootstrap.cc:492] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:01.805444 8545 tablet_bootstrap.cc:654] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.810818 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a7fa9f23989d4bdba6545d0e681c78e5
I20251028 03:00:01.811543 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet a7fa9f23989d4bdba6545d0e681c78e5 with delete_type TABLET_DATA_DELETED (replaced by tablet f9425f2912d34c9da2631056f7c43cc2) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:01.812145 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a7fa9f23989d4bdba6545d0e681c78e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7fa9f23989d4bdba6545d0e681c78e5 already in progress: creating tablet
I20251028 03:00:01.812842 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet a7fa9f23989d4bdba6545d0e681c78e5 with delete_type TABLET_DATA_DELETED (replaced by tablet f9425f2912d34c9da2631056f7c43cc2) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.812949 7961 catalog_manager.cc:6719] ReplaceTablet: tablet a7fa9f23989d4bdba6545d0e681c78e5 deleted and replaced by tablet f9425f2912d34c9da2631056f7c43cc2
I20251028 03:00:01.813309 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet a7fa9f23989d4bdba6545d0e681c78e5 with delete_type TABLET_DATA_DELETED (replaced by tablet f9425f2912d34c9da2631056f7c43cc2) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.813598 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_71a85f4535144a6b98c4ff7ab835e232" } new_table_name: "default.Table_01472823184b4a239a78ce504161be41" modify_external_catalogs: true
W20251028 03:00:01.813863 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a7fa9f23989d4bdba6545d0e681c78e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7fa9f23989d4bdba6545d0e681c78e5 already in progress: creating tablet
I20251028 03:00:01.814720 8544 tablet_bootstrap.cc:492] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:01.814863 8544 ts_tablet_manager.cc:1403] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 03:00:01.815480 8544 raft_consensus.cc:359] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.815639 8544 raft_consensus.cc:385] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.815707 8544 raft_consensus.cc:740] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.815603 8546 tablet_bootstrap.cc:654] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.815886 8544 consensus_queue.cc:260] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.819845 8544 ts_tablet_manager.cc:1434] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 03:00:01.820120 8544 tablet_bootstrap.cc:492] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:01.824899 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_806d85836a4c44a986095f5108a93d7f" } modify_external_catalogs: true
W20251028 03:00:01.825064 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ba40ba2c066e4c94885a595fd1c4445e consensus state has no leader
I20251028 03:00:01.825048 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_806d85836a4c44a986095f5108a93d7f" } modify_external_catalogs: true
I20251028 03:00:01.824900 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d9e8b9a5238c4b69ade009e5e8cabde7" } new_table_name: "default.Table_47b181d6668e4b35972c604a53761997" modify_external_catalogs: true
I20251028 03:00:01.825749 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 464e6b3483bc4816bfa844c52fd3ca53
I20251028 03:00:01.826627 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet 464e6b3483bc4816bfa844c52fd3ca53 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.826818 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 69e0358b593f4b55b6b4ec859e2c021d
I20251028 03:00:01.827056 7965 catalog_manager.cc:6719] ReplaceTablet: tablet 69e0358b593f4b55b6b4ec859e2c021d deleted and replaced by tablet 5a8e8f1a333240c2b40a0e59c3a04172
I20251028 03:00:01.827280 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 40a0431501d348b19ec89fe692741ce5 (table default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52]). Expected version 3 got 0
I20251028 03:00:01.827519 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 464e6b3483bc4816bfa844c52fd3ca53 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.828022 7966 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 40a0431501d348b19ec89fe692741ce5 (table default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52]). Expected version 3 got 0
I20251028 03:00:01.828374 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet 464e6b3483bc4816bfa844c52fd3ca53 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.828630 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet aa36c6870c1b4a459f0a41c84855a76e (table default.Table_d51065c1d57c4857bc5765d1baf9da06 [id=3815e9736bd34d668ec7f7e113e11e6f]). Expected version 1 got 0
I20251028 03:00:01.829743 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 69e0358b593f4b55b6b4ec859e2c021d with delete_type TABLET_DATA_DELETED (replaced by tablet 5a8e8f1a333240c2b40a0e59c3a04172) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.830075 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ff112c66f1a4cf9b232e6bb665755f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.830330 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bc54972260747f3af0c18d05c6a3442 with delete_type TABLET_DATA_DELETED (replaced by tablet 9d438110cfec42a991b6c40b0881cc62) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.830792 8544 tablet_bootstrap.cc:654] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.831705 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 69e0358b593f4b55b6b4ec859e2c021d with delete_type TABLET_DATA_DELETED (replaced by tablet 5a8e8f1a333240c2b40a0e59c3a04172) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.831900 8550 tablet_replica.cc:333] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.832037 8550 raft_consensus.cc:2243] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.832162 8550 raft_consensus.cc:2272] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.833078 8552 ts_tablet_manager.cc:1950] T 81b521c9f57844bb9873f6b26b5cf661 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.831441 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 69e0358b593f4b55b6b4ec859e2c021d with delete_type TABLET_DATA_DELETED (replaced by tablet 5a8e8f1a333240c2b40a0e59c3a04172) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.833892 8552 tablet_replica.cc:333] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.833943 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 81b521c9f57844bb9873f6b26b5cf661 (table default.Table_c14c90440d424269b7a05b90f49e57e6 [id=5f82b9596e5842deb4d45e19c91629bb]) successfully deleted
I20251028 03:00:01.834079 8552 raft_consensus.cc:2243] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
W20251028 03:00:01.834134 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a7fa9f23989d4bdba6545d0e681c78e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7fa9f23989d4bdba6545d0e681c78e5 already in progress: creating tablet
I20251028 03:00:01.834287 8552 raft_consensus.cc:2272] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.834022 8550 ts_tablet_manager.cc:1916] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.834800 8552 ts_tablet_manager.cc:1916] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.837198 8552 ts_tablet_manager.cc:1929] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.837277 8552 log.cc:1199] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/464e6b3483bc4816bfa844c52fd3ca53
I20251028 03:00:01.837548 8552 ts_tablet_manager.cc:1950] T 464e6b3483bc4816bfa844c52fd3ca53 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.838234 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 464e6b3483bc4816bfa844c52fd3ca53 (table default.Table_e95d99935d7d4d94a16f5a826847ce32 [id=71026b1300014ba6a66c3431c05cc86c]) successfully deleted
I20251028 03:00:01.839154 8550 ts_tablet_manager.cc:1929] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.839228 8550 log.cc:1199] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/464e6b3483bc4816bfa844c52fd3ca53
I20251028 03:00:01.839218 8551 tablet_replica.cc:333] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.839423 8551 raft_consensus.cc:2243] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.839563 8551 raft_consensus.cc:2272] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.839577 8545 tablet_bootstrap.cc:492] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.839666 8545 ts_tablet_manager.cc:1403] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.000s
I20251028 03:00:01.839762 8550 ts_tablet_manager.cc:1950] T 464e6b3483bc4816bfa844c52fd3ca53 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.840147 8545 raft_consensus.cc:359] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.840219 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 464e6b3483bc4816bfa844c52fd3ca53 (table default.Table_e95d99935d7d4d94a16f5a826847ce32 [id=71026b1300014ba6a66c3431c05cc86c]) successfully deleted
I20251028 03:00:01.840248 8545 raft_consensus.cc:385] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.840279 8545 raft_consensus.cc:740] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.840389 8545 consensus_queue.cc:260] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.840438 8550 tablet_replica.cc:333] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.840577 8550 raft_consensus.cc:2243] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.840648 8545 ts_tablet_manager.cc:1434] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.840687 8550 raft_consensus.cc:2272] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.840834 8545 tablet_bootstrap.cc:492] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.841060 8550 ts_tablet_manager.cc:1916] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.841975 8545 tablet_bootstrap.cc:654] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.842039 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a2e6ca8fa3354866af0f7f6133b3e5ea" } modify_external_catalogs: true
I20251028 03:00:01.842166 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a2e6ca8fa3354866af0f7f6133b3e5ea" } modify_external_catalogs: true
I20251028 03:00:01.843223 8550 ts_tablet_manager.cc:1929] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.844630 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5767228cc6d84aa0b3d29bee0c6cb362"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.845017 8550 log.cc:1199] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/69e0358b593f4b55b6b4ec859e2c021d
I20251028 03:00:01.845003 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_aa25a85ac8d24940a727a586d6c630db" } new_table_name: "default.Table_708ef26dc89446cfb5805c223cca9b20" modify_external_catalogs: true
W20251028 03:00:01.845059 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5767228cc6d84aa0b3d29bee0c6cb362 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.845319 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_56137eddf5dd461c8ef068d0dabb832c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.845640 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_56137eddf5dd461c8ef068d0dabb832c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.846300 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet f2d0dffdf5cc414ab870e0dde75916d6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b0c593a3b96434db9ac42d260fc6997) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.846455 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 4817e3b21c474b2589d30ef9823032d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.846380 8195 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:01.847491 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a4bcdc8a0c554b7cae81d7993e4f2861
I20251028 03:00:01.847927 8553 raft_consensus.cc:493] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.848021 8553 raft_consensus.cc:515] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.848457 8553 leader_election.cc:290] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:01.849090 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet a4bcdc8a0c554b7cae81d7993e4f2861 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.849095 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40a0431501d348b19ec89fe692741ce5" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:01.849220 8146 raft_consensus.cc:2468] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.849444 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_dc0c85038d58432db143e5af2d2f4dab" } new_table_name: "default.Table_cc6f477b10134718816a0594b9124205" modify_external_catalogs: true
I20251028 03:00:01.849545 8218 leader_election.cc:304] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:01.850064 9884 raft_consensus.cc:2804] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:01.850126 9884 raft_consensus.cc:493] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:01.850168 9884 raft_consensus.cc:3060] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.850368 8553 raft_consensus.cc:493] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.850451 8553 raft_consensus.cc:515] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.850791 8553 leader_election.cc:290] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:01.850870 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40a0431501d348b19ec89fe692741ce5" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.851011 8416 raft_consensus.cc:2468] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:01.851362 9884 raft_consensus.cc:515] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.851455 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2d0dffdf5cc414ab870e0dde75916d6" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.851766 9884 leader_election.cc:290] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 03:00:01.851816 8220 leader_election.cc:343] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.852280 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet a4bcdc8a0c554b7cae81d7993e4f2861 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.852569 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 746eae30ca284758a6e3ed8ad80c2cdc from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.852914 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40a0431501d348b19ec89fe692741ce5" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:01.853055 8144 raft_consensus.cc:3060] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.854085 8144 raft_consensus.cc:2468] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:01.854292 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet a4bcdc8a0c554b7cae81d7993e4f2861 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.854423 8218 leader_election.cc:304] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:01.854594 9884 raft_consensus.cc:2804] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:01.854674 9884 raft_consensus.cc:697] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:01.854826 9884 consensus_queue.cc:237] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.856017 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "40a0431501d348b19ec89fe692741ce5" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:01.856142 8416 raft_consensus.cc:3060] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:01.856323 8218 leader_election.cc:343] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: f2d0dffdf5cc414ab870e0dde75916d6
I20251028 03:00:01.856392 8218 leader_election.cc:304] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.856597 8553 raft_consensus.cc:2749] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.857244 8416 raft_consensus.cc:2468] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:01.858158 8545 tablet_bootstrap.cc:492] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.858278 8545 ts_tablet_manager.cc:1403] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 03:00:01.858795 8551 ts_tablet_manager.cc:1916] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.858826 8545 raft_consensus.cc:359] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.858919 8545 raft_consensus.cc:385] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.858947 8545 raft_consensus.cc:740] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.859097 8545 consensus_queue.cc:260] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.859840 8550 ts_tablet_manager.cc:1950] T 69e0358b593f4b55b6b4ec859e2c021d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.860278 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 69e0358b593f4b55b6b4ec859e2c021d (table default.Table_49f2ad3b1c934eaeab556927604ddf1b [id=e88ffc49924340aa92a3aedd631330bf]) successfully deleted
I20251028 03:00:01.860705 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet 4817e3b21c474b2589d30ef9823032d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:55 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.860774 8550 tablet_replica.cc:333] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.860929 8550 raft_consensus.cc:2243] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.861042 8550 raft_consensus.cc:2272] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.861578 8550 ts_tablet_manager.cc:1916] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.862643 8545 ts_tablet_manager.cc:1434] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 03:00:01.862888 8545 tablet_bootstrap.cc:492] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.862847 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_49fb24aa7f12420fae313eaa20ac98b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.864073 8545 tablet_bootstrap.cc:654] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.864215 8550 ts_tablet_manager.cc:1929] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.864282 8550 log.cc:1199] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2ff112c66f1a4cf9b232e6bb665755f9
I20251028 03:00:01.864482 8550 ts_tablet_manager.cc:1950] T 2ff112c66f1a4cf9b232e6bb665755f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.864903 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2ff112c66f1a4cf9b232e6bb665755f9 (table default.Table_9f07958f85cf4923a67bf8ac1ad1c722 [id=26d281a132204bc288eda2ac08794956]) successfully deleted
I20251028 03:00:01.865252 9190 raft_consensus.cc:493] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.865325 9190 raft_consensus.cc:515] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:01.865679 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 899cdb5e4671477db1e850244b7952dd has no consensus state
I20251028 03:00:01.865707 9190 leader_election.cc:290] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:01.866340 8354 leader_election.cc:343] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 4817e3b21c474b2589d30ef9823032d2
I20251028 03:00:01.866966 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4817e3b21c474b2589d30ef9823032d2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:01.867110 8273 raft_consensus.cc:2468] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:01.867419 8356 leader_election.cc:304] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.867676 9190 raft_consensus.cc:2804] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:01.867733 9190 raft_consensus.cc:493] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:01.867771 9190 raft_consensus.cc:3060] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:01.867786 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49fb24aa7f12420fae313eaa20ac98b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.868816 9190 raft_consensus.cc:515] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.869176 9190 leader_election.cc:290] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:01.869760 8354 leader_election.cc:343] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 4817e3b21c474b2589d30ef9823032d2
I20251028 03:00:01.869802 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4817e3b21c474b2589d30ef9823032d2" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:01.869756 8551 ts_tablet_manager.cc:1929] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.869905 8551 log.cc:1199] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/69e0358b593f4b55b6b4ec859e2c021d
I20251028 03:00:01.869922 8273 raft_consensus.cc:3060] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:01.870121 8551 ts_tablet_manager.cc:1950] T 69e0358b593f4b55b6b4ec859e2c021d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.870761 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 69e0358b593f4b55b6b4ec859e2c021d (table default.Table_49f2ad3b1c934eaeab556927604ddf1b [id=e88ffc49924340aa92a3aedd631330bf]) successfully deleted
I20251028 03:00:01.870973 8551 tablet_replica.cc:333] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.870947 8273 raft_consensus.cc:2468] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:01.871110 8551 raft_consensus.cc:2243] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.871209 8551 raft_consensus.cc:2272] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.871266 8356 leader_election.cc:304] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters: fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:01.871510 9190 raft_consensus.cc:2804] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:01.871590 9190 raft_consensus.cc:697] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:01.871714 9190 consensus_queue.cc:237] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.872303 8551 ts_tablet_manager.cc:1916] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:01.873307 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7ee583e318174f34b162c2a104a0a3a9 consensus state has no leader
I20251028 03:00:01.874936 8552 tablet_replica.cc:333] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.874976 8551 ts_tablet_manager.cc:1929] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.875066 8551 log.cc:1199] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/464e6b3483bc4816bfa844c52fd3ca53
I20251028 03:00:01.875324 8552 raft_consensus.cc:2243] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.875370 8551 ts_tablet_manager.cc:1950] T 464e6b3483bc4816bfa844c52fd3ca53 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.875576 8552 pending_rounds.cc:70] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 03:00:01.875777 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 464e6b3483bc4816bfa844c52fd3ca53 (table default.Table_e95d99935d7d4d94a16f5a826847ce32 [id=71026b1300014ba6a66c3431c05cc86c]) successfully deleted
I20251028 03:00:01.875665 8552 pending_rounds.cc:77] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597165526806528 op_type: NO_OP noop_request { }
I20251028 03:00:01.876106 8552 raft_consensus.cc:2889] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:01.876222 8552 raft_consensus.cc:2272] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.876857 8552 ts_tablet_manager.cc:1916] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.879469 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_aadf4f11e0134fc78dff262c2726bc8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.879922 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aadf4f11e0134fc78dff262c2726bc8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.882165 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet f2d0dffdf5cc414ab870e0dde75916d6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3b0c593a3b96434db9ac42d260fc6997) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.882776 8550 tablet_replica.cc:333] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.883008 8550 raft_consensus.cc:2243] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.883199 8550 raft_consensus.cc:2272] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.883898 8550 ts_tablet_manager.cc:1916] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:01.885412 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a4bcdc8a0c554b7cae81d7993e4f2861 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4bcdc8a0c554b7cae81d7993e4f2861 already in progress: creating tablet
I20251028 03:00:01.885720 8551 tablet_replica.cc:333] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.886536 8551 raft_consensus.cc:2243] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.886806 8551 pending_rounds.cc:70] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 03:00:01.886900 8551 pending_rounds.cc:77] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597166067961856 op_type: NO_OP noop_request { }
I20251028 03:00:01.887037 8551 raft_consensus.cc:2889] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:01.887042 8552 ts_tablet_manager.cc:1929] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.887118 8552 log.cc:1199] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/69e0358b593f4b55b6b4ec859e2c021d
I20251028 03:00:01.887254 8551 raft_consensus.cc:2272] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.887853 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4125bb9e525c4e7d91ae0862abc44584
I20251028 03:00:01.888305 8545 tablet_bootstrap.cc:492] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.888387 8545 ts_tablet_manager.cc:1403] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20251028 03:00:01.888355 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1b11f8e3f61d4362a6984c4ac6c9ff8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.888775 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b11f8e3f61d4362a6984c4ac6c9ff8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.888860 8551 ts_tablet_manager.cc:1916] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.888911 8545 raft_consensus.cc:359] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.889008 8545 raft_consensus.cc:385] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.889037 8545 raft_consensus.cc:740] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.889175 8545 consensus_queue.cc:260] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.890690 8545 ts_tablet_manager.cc:1434] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:01.890935 8545 tablet_bootstrap.cc:492] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.891083 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 4125bb9e525c4e7d91ae0862abc44584 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.892056 8545 tablet_bootstrap.cc:654] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.892768 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet 4125bb9e525c4e7d91ae0862abc44584 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.898814 8550 ts_tablet_manager.cc:1929] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.898926 8550 log.cc:1199] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/6bc54972260747f3af0c18d05c6a3442
I20251028 03:00:01.899165 8550 ts_tablet_manager.cc:1950] T 6bc54972260747f3af0c18d05c6a3442 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.899653 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet fbf0114b95a4492380d4048d8f4c97e4 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.900084 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 6bc54972260747f3af0c18d05c6a3442 (table default.Table_4fe6d656356041dcaad42a9fd6f026d5 [id=f8375f32144c4541bea5dbcd51ab256e]) successfully deleted
I20251028 03:00:01.900511 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c8239b64a71949bb8553a15b778102b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:01.900877 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8239b64a71949bb8553a15b778102b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.901441 8551 ts_tablet_manager.cc:1929] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:01.901525 8551 log.cc:1199] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4817e3b21c474b2589d30ef9823032d2
I20251028 03:00:01.901816 8551 ts_tablet_manager.cc:1950] T 4817e3b21c474b2589d30ef9823032d2 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 03:00:01.902274 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a4bcdc8a0c554b7cae81d7993e4f2861 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4bcdc8a0c554b7cae81d7993e4f2861 already in progress: creating tablet
I20251028 03:00:01.903105 8551 tablet_replica.cc:333] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.903224 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4817e3b21c474b2589d30ef9823032d2 (table default.Table_326908ab91a545829b75b06816614771 [id=06de831f38b74668aabbef36f08a4a12]) successfully deleted
I20251028 03:00:01.890221 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 4125bb9e525c4e7d91ae0862abc44584 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.903625 8552 ts_tablet_manager.cc:1950] T 69e0358b593f4b55b6b4ec859e2c021d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.904289 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 69e0358b593f4b55b6b4ec859e2c021d (table default.Table_49f2ad3b1c934eaeab556927604ddf1b [id=e88ffc49924340aa92a3aedd631330bf]) successfully deleted
I20251028 03:00:01.905001 8552 tablet_replica.cc:333] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.905184 8552 raft_consensus.cc:2243] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.905325 8552 raft_consensus.cc:2272] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.905664 8550 tablet_replica.cc:333] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.905943 8550 raft_consensus.cc:2243] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:01.906791 8550 raft_consensus.cc:2272] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.904844 8545 tablet_bootstrap.cc:492] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.907025 8545 ts_tablet_manager.cc:1403] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 03:00:01.907639 8545 raft_consensus.cc:359] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.907778 8545 raft_consensus.cc:385] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.907817 8545 raft_consensus.cc:740] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.907950 8545 consensus_queue.cc:260] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:01.908687 8545 ts_tablet_manager.cc:1434] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:01.909545 8468 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:01.909883 8545 tablet_bootstrap.cc:492] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.912179 8545 tablet_bootstrap.cc:654] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.912331 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_121342cd2f774cce941d56a09004daae" } modify_external_catalogs: true
I20251028 03:00:01.912448 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_121342cd2f774cce941d56a09004daae" } modify_external_catalogs: true
W20251028 03:00:01.912789 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 38ad9a5b7f0d46d29746d7ed1f8ffda2 consensus state has no leader
I20251028 03:00:01.912976 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c6a86fd3f7c140eb852bd8a9bd48f8ea" } modify_external_catalogs: true
I20251028 03:00:01.913084 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c6a86fd3f7c140eb852bd8a9bd48f8ea" } modify_external_catalogs: true
I20251028 03:00:01.913637 8551 raft_consensus.cc:2243] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.913784 8551 raft_consensus.cc:2272] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.908537 9788 raft_consensus.cc:493] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:01.913944 9788 raft_consensus.cc:515] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.914359 9788 leader_election.cc:290] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:01.914721 8146 raft_consensus.cc:1275] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:01.914791 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "338cf9d02142436f97ccd44313cc8199" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:01.915066 8416 raft_consensus.cc:1275] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20251028 03:00:01.915130 8083 leader_election.cc:343] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.915269 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "338cf9d02142436f97ccd44313cc8199" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:01.915529 9884 consensus_queue.cc:1048] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 03:00:01.915866 8083 leader_election.cc:343] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:01.915925 8083 leader_election.cc:304] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:01.916064 9788 raft_consensus.cc:2749] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:01.916505 8550 ts_tablet_manager.cc:1916] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.918951 8551 ts_tablet_manager.cc:1916] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.919267 9884 consensus_queue.cc:1048] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:01.920107 7969 catalog_manager.cc:5649] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:01.920307 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 40a0431501d348b19ec89fe692741ce5 (table default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52]). Expected version 3 got 0
I20251028 03:00:01.924655 8259 tablet_service.cc:1505] Processing CreateTablet for tablet ce9b928270f54a2d97711280cf7259ad (DEFAULT_TABLE table=default.Table_1c980a2eaf3c4a458846ce8e022c35d4 [id=0da34ae93aef4751b7c11a36a5a6ee05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.924989 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce9b928270f54a2d97711280cf7259ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.925029 8122 tablet_service.cc:1505] Processing CreateTablet for tablet ce9b928270f54a2d97711280cf7259ad (DEFAULT_TABLE table=default.Table_1c980a2eaf3c4a458846ce8e022c35d4 [id=0da34ae93aef4751b7c11a36a5a6ee05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.925194 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 44fe3a7cdd1d4c76888df499d929977a (DEFAULT_TABLE table=default.Table_9a9b719b65a54f52af68c68c87bb18a3 [id=2c35ac28ac284310ad2a79377f9f5b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.925280 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce9b928270f54a2d97711280cf7259ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.925509 8390 tablet_service.cc:1505] Processing CreateTablet for tablet ce9b928270f54a2d97711280cf7259ad (DEFAULT_TABLE table=default.Table_1c980a2eaf3c4a458846ce8e022c35d4 [id=0da34ae93aef4751b7c11a36a5a6ee05]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.927546 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 44fe3a7cdd1d4c76888df499d929977a (DEFAULT_TABLE table=default.Table_9a9b719b65a54f52af68c68c87bb18a3 [id=2c35ac28ac284310ad2a79377f9f5b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.927806 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44fe3a7cdd1d4c76888df499d929977a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.930229 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 44fe3a7cdd1d4c76888df499d929977a (DEFAULT_TABLE table=default.Table_9a9b719b65a54f52af68c68c87bb18a3 [id=2c35ac28ac284310ad2a79377f9f5b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.930512 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44fe3a7cdd1d4c76888df499d929977a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.931088 8386 tablet_service.cc:1505] Processing CreateTablet for tablet d3c71268fc76499e905317057af15a75 (DEFAULT_TABLE table=default.Table_d3251b07042c44ca9807ffa0df1274fc [id=628e425682a64331920dee76bd1126fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.932500 8387 tablet_service.cc:1505] Processing CreateTablet for tablet af93c341f684426fa8a847ee06d86d35 (DEFAULT_TABLE table=default.Table_c8a55dcfb9e94100866bac84b2ce9ecd [id=a87c19bc933b49b5bbbe15b571bc13cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.932500 8396 tablet_service.cc:1505] Processing CreateTablet for tablet e105f414dbb8487195ab201cfa914dec (DEFAULT_TABLE table=default.Table_efe45b07a49349e7a7995ee1b0c4993e [id=2b5fa0c636b74763937e8728d8e87b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.932713 8120 tablet_service.cc:1505] Processing CreateTablet for tablet af93c341f684426fa8a847ee06d86d35 (DEFAULT_TABLE table=default.Table_c8a55dcfb9e94100866bac84b2ce9ecd [id=a87c19bc933b49b5bbbe15b571bc13cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.932852 8118 tablet_service.cc:1505] Processing CreateTablet for tablet d3c71268fc76499e905317057af15a75 (DEFAULT_TABLE table=default.Table_d3251b07042c44ca9807ffa0df1274fc [id=628e425682a64331920dee76bd1126fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.932945 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af93c341f684426fa8a847ee06d86d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.933071 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c71268fc76499e905317057af15a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.934024 8260 tablet_service.cc:1505] Processing CreateTablet for tablet af93c341f684426fa8a847ee06d86d35 (DEFAULT_TABLE table=default.Table_c8a55dcfb9e94100866bac84b2ce9ecd [id=a87c19bc933b49b5bbbe15b571bc13cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.934284 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af93c341f684426fa8a847ee06d86d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.935179 8112 tablet_service.cc:1505] Processing CreateTablet for tablet e105f414dbb8487195ab201cfa914dec (DEFAULT_TABLE table=default.Table_efe45b07a49349e7a7995ee1b0c4993e [id=2b5fa0c636b74763937e8728d8e87b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.935431 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e105f414dbb8487195ab201cfa914dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.935601 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet fbf0114b95a4492380d4048d8f4c97e4
I20251028 03:00:01.935879 7968 catalog_manager.cc:6719] ReplaceTablet: tablet fbf0114b95a4492380d4048d8f4c97e4 deleted and replaced by tablet fcc757c38ac6492e9697bcdd053364c6
I20251028 03:00:01.937209 8263 tablet_service.cc:1505] Processing CreateTablet for tablet d3c71268fc76499e905317057af15a75 (DEFAULT_TABLE table=default.Table_d3251b07042c44ca9807ffa0df1274fc [id=628e425682a64331920dee76bd1126fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.937256 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet fbf0114b95a4492380d4048d8f4c97e4 with delete_type TABLET_DATA_DELETED (replaced by tablet fcc757c38ac6492e9697bcdd053364c6) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.937470 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c71268fc76499e905317057af15a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.937675 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet fbf0114b95a4492380d4048d8f4c97e4 with delete_type TABLET_DATA_DELETED (replaced by tablet fcc757c38ac6492e9697bcdd053364c6) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.938306 8392 tablet_service.cc:1505] Processing CreateTablet for tablet 5ce070da42d94fdba260478115c617a9 (DEFAULT_TABLE table=default.Table_3fb2a054809d4428abf6fd4cee9a79cf [id=7b116512f15444c2a31c644616c5fbab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.938771 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 5ce070da42d94fdba260478115c617a9 (DEFAULT_TABLE table=default.Table_3fb2a054809d4428abf6fd4cee9a79cf [id=7b116512f15444c2a31c644616c5fbab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.938989 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ce070da42d94fdba260478115c617a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.941378 8389 tablet_service.cc:1505] Processing CreateTablet for tablet 599e141929f54fd9a1658f9c247d8f7e (DEFAULT_TABLE table=default.Table_5d3da0f4d8af4dffb109a35707fe3a8f [id=06160e2b13264aefac8884c87b2a7898]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.941591 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet fbf0114b95a4492380d4048d8f4c97e4 with delete_type TABLET_DATA_DELETED (replaced by tablet fcc757c38ac6492e9697bcdd053364c6) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.941815 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 5ce070da42d94fdba260478115c617a9 (DEFAULT_TABLE table=default.Table_3fb2a054809d4428abf6fd4cee9a79cf [id=7b116512f15444c2a31c644616c5fbab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.941951 8259 tablet_service.cc:1505] Processing CreateTablet for tablet e105f414dbb8487195ab201cfa914dec (DEFAULT_TABLE table=default.Table_efe45b07a49349e7a7995ee1b0c4993e [id=2b5fa0c636b74763937e8728d8e87b19]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.942188 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e105f414dbb8487195ab201cfa914dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.943344 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 11f8b9bffa3048bdae44d3eaf25cf1f1 (DEFAULT_TABLE table=default.Table_aa11196c63eb4ee4ba2c3597325d460a [id=6417551cacf044e29dc79e8353579fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.944113 7967 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 338cf9d02142436f97ccd44313cc8199 (table default.Table_77a37bbdfa4f41458e6b50fb213850bc [id=5f61b3c12ed647968fc2932aa24a13f3]). Expected version 2 got 0
I20251028 03:00:01.944782 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 11f8b9bffa3048bdae44d3eaf25cf1f1 (DEFAULT_TABLE table=default.Table_aa11196c63eb4ee4ba2c3597325d460a [id=6417551cacf044e29dc79e8353579fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.945030 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f8b9bffa3048bdae44d3eaf25cf1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.946244 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 599e141929f54fd9a1658f9c247d8f7e (DEFAULT_TABLE table=default.Table_5d3da0f4d8af4dffb109a35707fe3a8f [id=06160e2b13264aefac8884c87b2a7898]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.946408 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 12d103e9b5024a87aa232c26e0c51ab9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.947125 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ce070da42d94fdba260478115c617a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.947364 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 599e141929f54fd9a1658f9c247d8f7e (DEFAULT_TABLE table=default.Table_5d3da0f4d8af4dffb109a35707fe3a8f [id=06160e2b13264aefac8884c87b2a7898]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:01.947582 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 599e141929f54fd9a1658f9c247d8f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.948359 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet b5e3c89222754eacb6ba4773396d62a9 (table default.Table_708ef26dc89446cfb5805c223cca9b20 [id=10f40fa1652f495e91fcbd5852bc3464]). Expected version 1 got 0
I20251028 03:00:01.948928 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 599e141929f54fd9a1658f9c247d8f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.952469 8550 ts_tablet_manager.cc:1929] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.952852 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44fe3a7cdd1d4c76888df499d929977a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.952981 8550 log.cc:1199] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4125bb9e525c4e7d91ae0862abc44584
I20251028 03:00:01.953023 8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 599e141929f54fd9a1658f9c247d8f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.953204 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce9b928270f54a2d97711280cf7259ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.953384 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ce070da42d94fdba260478115c617a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.953497 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e105f414dbb8487195ab201cfa914dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.953692 8550 ts_tablet_manager.cc:1950] T 4125bb9e525c4e7d91ae0862abc44584 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.953471 8551 ts_tablet_manager.cc:1929] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.953855 8551 log.cc:1199] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/f2d0dffdf5cc414ab870e0dde75916d6
I20251028 03:00:01.956039 8552 ts_tablet_manager.cc:1916] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.955396 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af93c341f684426fa8a847ee06d86d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.958328 8551 ts_tablet_manager.cc:1950] T f2d0dffdf5cc414ab870e0dde75916d6 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.958423 8552 ts_tablet_manager.cc:1929] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.958494 8552 log.cc:1199] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/f2d0dffdf5cc414ab870e0dde75916d6
I20251028 03:00:01.958683 8552 ts_tablet_manager.cc:1950] T f2d0dffdf5cc414ab870e0dde75916d6 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.959025 8552 tablet_replica.cc:333] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.959019 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet f2d0dffdf5cc414ab870e0dde75916d6 (table default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]) successfully deleted
I20251028 03:00:01.959162 8552 raft_consensus.cc:2243] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.959601 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4125bb9e525c4e7d91ae0862abc44584 (table default.Table_a2e6ca8fa3354866af0f7f6133b3e5ea [id=3131dc70f17c4cdf84d08ab26c718e78]) successfully deleted
I20251028 03:00:01.959718 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet f2d0dffdf5cc414ab870e0dde75916d6 (table default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]) successfully deleted
I20251028 03:00:01.959271 8552 raft_consensus.cc:2272] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.959909 8550 tablet_replica.cc:333] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:01.960089 8550 raft_consensus.cc:2243] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.960285 8551 tablet_replica.cc:333] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:01.960443 8550 raft_consensus.cc:2272] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.960462 8552 ts_tablet_manager.cc:1916] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:01.960721 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet fbf0114b95a4492380d4048d8f4c97e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbf0114b95a4492380d4048d8f4c97e4 already in progress: creating tablet
I20251028 03:00:01.960804 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c71268fc76499e905317057af15a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.960960 8551 raft_consensus.cc:2243] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.961092 8551 raft_consensus.cc:2272] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.961616 8550 ts_tablet_manager.cc:1916] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.962494 8552 ts_tablet_manager.cc:1929] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.962432 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_be36f6bb04274166a890026a70c7dbf7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:01.962702 8552 log.cc:1199] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4817e3b21c474b2589d30ef9823032d2
W20251028 03:00:01.962877 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be36f6bb04274166a890026a70c7dbf7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:01.963119 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f8b9bffa3048bdae44d3eaf25cf1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:01.963398 8552 ts_tablet_manager.cc:1950] T 4817e3b21c474b2589d30ef9823032d2 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.963454 8551 ts_tablet_manager.cc:1916] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.963887 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4817e3b21c474b2589d30ef9823032d2 (table default.Table_326908ab91a545829b75b06816614771 [id=06de831f38b74668aabbef36f08a4a12]) successfully deleted
W20251028 03:00:01.964354 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a4bcdc8a0c554b7cae81d7993e4f2861 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4bcdc8a0c554b7cae81d7993e4f2861 already in progress: creating tablet
I20251028 03:00:01.964558 8552 tablet_replica.cc:333] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:01.964751 8552 raft_consensus.cc:2243] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:01.964922 8552 raft_consensus.cc:2272] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:01.966655 8550 ts_tablet_manager.cc:1929] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:01.967605 8552 ts_tablet_manager.cc:1916] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:01.967969 8551 ts_tablet_manager.cc:1929] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.968055 8551 log.cc:1199] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4125bb9e525c4e7d91ae0862abc44584
I20251028 03:00:01.968320 8551 ts_tablet_manager.cc:1950] T 4125bb9e525c4e7d91ae0862abc44584 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:01.968793 8550 log.cc:1199] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/12d103e9b5024a87aa232c26e0c51ab9
I20251028 03:00:01.968986 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ce4cb3813e7045fe9eb474052d1bdf1a
I20251028 03:00:01.969094 8550 ts_tablet_manager.cc:1950] T 12d103e9b5024a87aa232c26e0c51ab9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:01.969866 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet ce4cb3813e7045fe9eb474052d1bdf1a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.969871 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet ce4cb3813e7045fe9eb474052d1bdf1a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.970484 8552 ts_tablet_manager.cc:1929] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:01.971251 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 12d103e9b5024a87aa232c26e0c51ab9 (table default.Table_12b6b08740be470382334a74e5df73cb [id=1b3819ae7b6f4269a5be015632d5c786]) successfully deleted
I20251028 03:00:01.971522 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet ce4cb3813e7045fe9eb474052d1bdf1a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:01.971604 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4125bb9e525c4e7d91ae0862abc44584 (table default.Table_a2e6ca8fa3354866af0f7f6133b3e5ea [id=3131dc70f17c4cdf84d08ab26c718e78]) successfully deleted
W20251028 03:00:01.972038 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet fbf0114b95a4492380d4048d8f4c97e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbf0114b95a4492380d4048d8f4c97e4 already in progress: creating tablet
W20251028 03:00:01.972419 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ce4cb3813e7045fe9eb474052d1bdf1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce4cb3813e7045fe9eb474052d1bdf1a already in progress: creating tablet
W20251028 03:00:01.972913 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ce4cb3813e7045fe9eb474052d1bdf1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce4cb3813e7045fe9eb474052d1bdf1a already in progress: creating tablet
I20251028 03:00:01.973068 8545 tablet_bootstrap.cc:492] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:01.973165 8545 ts_tablet_manager.cc:1403] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.063s user 0.002s sys 0.000s
I20251028 03:00:01.973610 8545 raft_consensus.cc:359] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.973712 8545 raft_consensus.cc:385] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:01.973742 8545 raft_consensus.cc:740] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:01.973841 8545 consensus_queue.cc:260] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:01.974116 8545 ts_tablet_manager.cc:1434] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:01.974295 8545 tablet_bootstrap.cc:492] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:01.978711 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 37a128fff52e4c9d9e1cd863fe80ec9c from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:01.979382 9909 tablet.cc:1722] T b5e3c89222754eacb6ba4773396d62a9 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:01.972860 10187 tablet.cc:1722] T b5e3c89222754eacb6ba4773396d62a9 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:01.975279 8545 tablet_bootstrap.cc:654] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:01.976321 10192 tablet.cc:1722] T b5e3c89222754eacb6ba4773396d62a9 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:01.984943 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd
I20251028 03:00:01.985836 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:01.986233 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:01.986289 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:01.986963 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd already in progress: creating tablet
W20251028 03:00:01.987123 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd already in progress: creating tablet
I20251028 03:00:01.989351 8552 log.cc:1199] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4125bb9e525c4e7d91ae0862abc44584
I20251028 03:00:01.989758 8552 ts_tablet_manager.cc:1950] T 4125bb9e525c4e7d91ae0862abc44584 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:01.990686 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (table default.Table_93b5aac358764b869e8589ccc023ea0e [id=674f76f9afed4f3f9e2c165e2479d938]). Expected version 1 got 0
I20251028 03:00:01.990931 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet b5e3c89222754eacb6ba4773396d62a9 (table default.Table_708ef26dc89446cfb5805c223cca9b20 [id=10f40fa1652f495e91fcbd5852bc3464]). Expected version 1 got 0
I20251028 03:00:01.991223 7967 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 2bf100407b034626b4a9f7e2340f9cf0 (table default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262]). Expected version 2 got 0
I20251028 03:00:01.992413 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet b5e3c89222754eacb6ba4773396d62a9 (table default.Table_708ef26dc89446cfb5805c223cca9b20 [id=10f40fa1652f495e91fcbd5852bc3464]). Expected version 1 got 0
I20251028 03:00:01.994150 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4125bb9e525c4e7d91ae0862abc44584 (table default.Table_a2e6ca8fa3354866af0f7f6133b3e5ea [id=3131dc70f17c4cdf84d08ab26c718e78]) successfully deleted
W20251028 03:00:01.994618 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet fbf0114b95a4492380d4048d8f4c97e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fbf0114b95a4492380d4048d8f4c97e4 already in progress: creating tablet
W20251028 03:00:01.997200 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ce4cb3813e7045fe9eb474052d1bdf1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce4cb3813e7045fe9eb474052d1bdf1a already in progress: creating tablet
W20251028 03:00:01.999408 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f6aeb23d2fa14d03adbbe7c9d54ae9fd already in progress: creating tablet
I20251028 03:00:02.002826 8546 tablet_bootstrap.cc:492] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.003013 8546 ts_tablet_manager.cc:1403] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.199s user 0.001s sys 0.002s
I20251028 03:00:02.003508 8545 tablet_bootstrap.cc:492] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.003593 8545 ts_tablet_manager.cc:1403] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.002s
I20251028 03:00:02.003700 8546 raft_consensus.cc:359] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.003815 8546 raft_consensus.cc:385] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.003854 8546 raft_consensus.cc:740] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.004031 8545 raft_consensus.cc:359] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.004040 8546 consensus_queue.cc:260] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.004127 8545 raft_consensus.cc:385] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.004155 8545 raft_consensus.cc:740] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.004261 8545 consensus_queue.cc:260] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.004453 8546 ts_tablet_manager.cc:1434] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.004544 8545 ts_tablet_manager.cc:1434] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:02.004643 8546 tablet_bootstrap.cc:492] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.004707 8545 tablet_bootstrap.cc:492] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.005656 8545 tablet_bootstrap.cc:654] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.005820 8546 tablet_bootstrap.cc:654] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.009259 9190 raft_consensus.cc:493] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.009352 9190 raft_consensus.cc:515] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.009727 9190 leader_election.cc:290] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.010115 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 11f8b9bffa3048bdae44d3eaf25cf1f1 (DEFAULT_TABLE table=default.Table_aa11196c63eb4ee4ba2c3597325d460a [id=6417551cacf044e29dc79e8353579fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.010334 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "626c2cc2d19047b794bf16c1b078941f" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.010421 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f8b9bffa3048bdae44d3eaf25cf1f1. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:02.010448 8354 leader_election.cc:343] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 626c2cc2d19047b794bf16c1b078941f
W20251028 03:00:02.010691 8356 leader_election.cc:343] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.010771 8356 leader_election.cc:304] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:02.011132 9190 raft_consensus.cc:2749] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.012315 8545 tablet_bootstrap.cc:492] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.012404 8545 ts_tablet_manager.cc:1403] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20251028 03:00:02.013032 8545 raft_consensus.cc:359] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.013139 8545 raft_consensus.cc:385] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.013172 8545 raft_consensus.cc:740] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.013285 8545 consensus_queue.cc:260] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.013586 8545 ts_tablet_manager.cc:1434] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:02.013862 8545 tablet_bootstrap.cc:492] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.016347 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 37a128fff52e4c9d9e1cd863fe80ec9c
I20251028 03:00:02.016672 7966 catalog_manager.cc:6719] ReplaceTablet: tablet 37a128fff52e4c9d9e1cd863fe80ec9c deleted and replaced by tablet 6d238a79585e481e97efee4ffd082421
I20251028 03:00:02.016940 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 37a128fff52e4c9d9e1cd863fe80ec9c with delete_type TABLET_DATA_DELETED (replaced by tablet 6d238a79585e481e97efee4ffd082421) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.017119 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 37a128fff52e4c9d9e1cd863fe80ec9c with delete_type TABLET_DATA_DELETED (replaced by tablet 6d238a79585e481e97efee4ffd082421) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:02.017642 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 37a128fff52e4c9d9e1cd863fe80ec9c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37a128fff52e4c9d9e1cd863fe80ec9c already in progress: creating tablet
W20251028 03:00:02.017795 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 37a128fff52e4c9d9e1cd863fe80ec9c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37a128fff52e4c9d9e1cd863fe80ec9c already in progress: creating tablet
I20251028 03:00:02.019924 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 37a128fff52e4c9d9e1cd863fe80ec9c with delete_type TABLET_DATA_DELETED (replaced by tablet 6d238a79585e481e97efee4ffd082421) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.021597 8545 tablet_bootstrap.cc:654] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:02.022068 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 37a128fff52e4c9d9e1cd863fe80ec9c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37a128fff52e4c9d9e1cd863fe80ec9c already in progress: creating tablet
I20251028 03:00:02.024232 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 8ab9ce61461d4fbc8466e61ebb184848 (DEFAULT_TABLE table=default.Table_3ea2d5c9d6d640ab90388e8696ebc2a6 [id=a2399ccdc83f494cba43c7a6a4b7a097]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.024502 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab9ce61461d4fbc8466e61ebb184848. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.024919 8544 tablet_bootstrap.cc:492] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:02.025039 8544 ts_tablet_manager.cc:1403] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.205s user 0.003s sys 0.000s
I20251028 03:00:02.025557 8544 raft_consensus.cc:359] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.025687 8544 raft_consensus.cc:385] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.025728 8544 raft_consensus.cc:740] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.025852 8544 consensus_queue.cc:260] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.026187 8544 ts_tablet_manager.cc:1434] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.026399 8544 tablet_bootstrap.cc:492] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:02.027838 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_149a395a74c243f29f343d30b3cf6bb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.028244 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_149a395a74c243f29f343d30b3cf6bb6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.030095 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_8a28a3e233bb4011b36e95509d454cc6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.030494 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a28a3e233bb4011b36e95509d454cc6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.031224 8545 tablet_bootstrap.cc:492] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.031323 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f5fe354dcfca4c57b4102be932490256" } modify_external_catalogs: true
I20251028 03:00:02.031435 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f5fe354dcfca4c57b4102be932490256" } modify_external_catalogs: true
I20251028 03:00:02.031394 8545 ts_tablet_manager.cc:1403] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.002s
I20251028 03:00:02.032016 8545 raft_consensus.cc:359] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.032183 8545 raft_consensus.cc:385] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.032255 8545 raft_consensus.cc:740] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.032590 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5008c46b0dcb452497e6dbfd59c37b92" } new_table_name: "default.Table_72c9968c521b4660aa662417c6857bf5" modify_external_catalogs: true
I20251028 03:00:02.032879 8545 consensus_queue.cc:260] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.033515 8545 ts_tablet_manager.cc:1434] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:02.034480 8545 tablet_bootstrap.cc:492] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.036821 8544 tablet_bootstrap.cc:654] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.041874 8545 tablet_bootstrap.cc:654] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.043454 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f43f1b75476441f294e6ca3839d1591f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:02.043740 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 8ab9ce61461d4fbc8466e61ebb184848 (DEFAULT_TABLE table=default.Table_3ea2d5c9d6d640ab90388e8696ebc2a6 [id=a2399ccdc83f494cba43c7a6a4b7a097]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 03:00:02.043870 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f43f1b75476441f294e6ca3839d1591f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.044056 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab9ce61461d4fbc8466e61ebb184848. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.047209 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 8ab9ce61461d4fbc8466e61ebb184848 (DEFAULT_TABLE table=default.Table_3ea2d5c9d6d640ab90388e8696ebc2a6 [id=a2399ccdc83f494cba43c7a6a4b7a097]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.047572 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab9ce61461d4fbc8466e61ebb184848. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.048838 7962 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c061797d60f14b298c963858268b5e60" } modify_external_catalogs: true
I20251028 03:00:02.048964 7962 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c061797d60f14b298c963858268b5e60" } modify_external_catalogs: true
I20251028 03:00:02.053051 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet 8e202e53053248a7a9cee6a217999489 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:02.056254 8387 tablet_service.cc:1505] Processing CreateTablet for tablet ed8b4bb7b8774e308d0f4744bae8f607 (DEFAULT_TABLE table=default.Table_5f3e79b2b1494fe49fae9db68cd963d8 [id=cc53786745144900a52d08a57573796f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.056578 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed8b4bb7b8774e308d0f4744bae8f607. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.056650 8545 tablet_bootstrap.cc:492] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.056728 8545 ts_tablet_manager.cc:1403] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.003s
I20251028 03:00:02.057304 8545 raft_consensus.cc:359] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.057500 8545 raft_consensus.cc:385] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.057668 8545 raft_consensus.cc:740] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.058022 8545 consensus_queue.cc:260] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.060089 7964 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_708ef26dc89446cfb5805c223cca9b20 [id=10f40fa1652f495e91fcbd5852bc3464] alter complete (version 1)
I20251028 03:00:02.061147 8263 tablet_service.cc:1505] Processing CreateTablet for tablet ed8b4bb7b8774e308d0f4744bae8f607 (DEFAULT_TABLE table=default.Table_5f3e79b2b1494fe49fae9db68cd963d8 [id=cc53786745144900a52d08a57573796f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.061419 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed8b4bb7b8774e308d0f4744bae8f607. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.064746 8545 ts_tablet_manager.cc:1434] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20251028 03:00:02.067982 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a96f9bde439b4ee0a079bb871e47b3a8" } new_table_name: "default.Table_7fb659aacea847c180ab8750d2fc1292" modify_external_catalogs: true
I20251028 03:00:02.068766 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 635f6166690f4452a5cf05f9e9a0f253 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:02.070719 8544 tablet_bootstrap.cc:492] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:02.070834 8544 ts_tablet_manager.cc:1403] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.000s
I20251028 03:00:02.070987 8545 tablet_bootstrap.cc:492] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.071416 8544 raft_consensus.cc:359] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.071537 8544 raft_consensus.cc:385] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.071580 8544 raft_consensus.cc:740] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.071930 8544 consensus_queue.cc:260] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.072095 8545 tablet_bootstrap.cc:654] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.072314 8544 ts_tablet_manager.cc:1434] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.072889 7969 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (table default.Table_93b5aac358764b869e8589ccc023ea0e [id=674f76f9afed4f3f9e2c165e2479d938]). Expected version 1 got 0
I20251028 03:00:02.073755 8544 tablet_bootstrap.cc:492] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:02.075340 8544 tablet_bootstrap.cc:654] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.078939 8416 raft_consensus.cc:3060] T 68ab2fe9132441c1834aea22561f02e9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.080567 8553 consensus_queue.cc:1048] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.081444 7963 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 5c1994484abf4b6fbbc3f47527558cdb (table default.Table_f454943605324eec83a70310f675f27f [id=b0407c8f78a84245a954c67e888da3ec]). Expected version 4 got 0
I20251028 03:00:02.083458 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_71dd113f5d12423d9aca617205523aa1" } new_table_name: "default.Table_8a0f22f6a841445e9db49817cc493481" modify_external_catalogs: true
I20251028 03:00:02.083554 8468 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:02.084913 8112 tablet_service.cc:1505] Processing CreateTablet for tablet ed8b4bb7b8774e308d0f4744bae8f607 (DEFAULT_TABLE table=default.Table_5f3e79b2b1494fe49fae9db68cd963d8 [id=cc53786745144900a52d08a57573796f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.085328 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed8b4bb7b8774e308d0f4744bae8f607. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.092187 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_00e05a1a71944e26a7e93167c3fdce40" } modify_external_catalogs: true
I20251028 03:00:02.092347 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_00e05a1a71944e26a7e93167c3fdce40" } modify_external_catalogs: true
I20251028 03:00:02.093724 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 9a81841c22ae49c0a17931d90407accc (table default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]). Expected version 1 got 0
I20251028 03:00:02.093951 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet f422030a6a4042009efc7fcfb21a6bca (table default.Table_a01151cafd944efcbc12e5f654a7a05a [id=fdd6ecd598f240948882d710fd485da2]). Expected version 2 got 0
I20251028 03:00:02.095259 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.095485 8551 tablet_replica.cc:333] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:02.095639 8551 raft_consensus.cc:2243] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.095759 8551 raft_consensus.cc:2272] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.095865 8545 tablet_bootstrap.cc:492] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.096277 8551 ts_tablet_manager.cc:1916] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.098583 8551 ts_tablet_manager.cc:1929] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.098644 8551 log.cc:1199] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9277a4ffd0ca4bf7afd29e1c6ab33bc9
I20251028 03:00:02.098838 8551 ts_tablet_manager.cc:1950] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:02.102834 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2687d010e9e444698c450ff0db40f8da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.103263 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2687d010e9e444698c450ff0db40f8da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.103255 9884 consensus_queue.cc:1048] T 68ab2fe9132441c1834aea22561f02e9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.105839 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 (table default.Table_eca57c87ce0240a0844cc210e4dbdd85 [id=289693853ee848e785d5460a21ae5063]) successfully deleted
I20251028 03:00:02.107532 8545 ts_tablet_manager.cc:1403] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.037s user 0.000s sys 0.003s
I20251028 03:00:02.108150 8545 raft_consensus.cc:359] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.109565 8545 raft_consensus.cc:385] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.109692 8545 raft_consensus.cc:740] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.110217 8545 consensus_queue.cc:260] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.111702 8545 ts_tablet_manager.cc:1434] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20251028 03:00:02.112062 8545 tablet_bootstrap.cc:492] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.113780 8545 tablet_bootstrap.cc:654] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.115162 9840 raft_consensus.cc:493] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.115248 9840 raft_consensus.cc:515] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.115686 9840 leader_election.cc:290] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.116452 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba40ba2c066e4c94885a595fd1c4445e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.117023 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba40ba2c066e4c94885a595fd1c4445e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.117158 8146 raft_consensus.cc:2468] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.119019 8354 leader_election.cc:304] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:02.119532 9840 raft_consensus.cc:2804] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.119601 9840 raft_consensus.cc:493] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.119657 9840 raft_consensus.cc:3060] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.120661 9840 raft_consensus.cc:515] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.121028 9840 leader_election.cc:290] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.122282 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba40ba2c066e4c94885a595fd1c4445e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.122406 8146 raft_consensus.cc:3060] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.122888 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba40ba2c066e4c94885a595fd1c4445e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:02.122947 8356 leader_election.cc:343] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:02.123245 8356 leader_election.cc:343] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.123564 8146 raft_consensus.cc:2468] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.123945 8354 leader_election.cc:304] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.124341 9840 raft_consensus.cc:2804] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.124432 9840 raft_consensus.cc:697] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:02.124562 9840 consensus_queue.cc:237] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.133371 9788 raft_consensus.cc:493] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.133507 9788 raft_consensus.cc:515] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.133953 9788 leader_election.cc:290] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.134331 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e45fe8888fbc415ea637b363dccdd06b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.134411 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e45fe8888fbc415ea637b363dccdd06b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:02.134776 8083 leader_election.cc:343] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:02.135035 8083 leader_election.cc:343] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.135092 8083 leader_election.cc:304] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:02.135596 9788 raft_consensus.cc:2749] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.136649 8546 tablet_bootstrap.cc:492] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.136757 8546 ts_tablet_manager.cc:1403] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.132s user 0.002s sys 0.000s
I20251028 03:00:02.137419 8546 raft_consensus.cc:359] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.137532 8546 raft_consensus.cc:385] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.137578 8546 raft_consensus.cc:740] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.137709 8546 consensus_queue.cc:260] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.137923 9958 raft_consensus.cc:493] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.138001 9958 raft_consensus.cc:515] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.138388 9958 leader_election.cc:290] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:02.138809 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2bf100407b034626b4a9f7e2340f9cf0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.138942 8146 raft_consensus.cc:2468] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:02.139168 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2bf100407b034626b4a9f7e2340f9cf0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:02.139230 8218 leader_election.cc:304] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:02.139287 8416 raft_consensus.cc:2468] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:02.139525 9958 raft_consensus.cc:2804] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.139611 9958 raft_consensus.cc:493] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.139653 9958 raft_consensus.cc:3060] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.140739 9958 raft_consensus.cc:515] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.141117 9958 leader_election.cc:290] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:02.142174 8546 ts_tablet_manager.cc:1434] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 03:00:02.142377 8546 tablet_bootstrap.cc:492] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.142606 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2bf100407b034626b4a9f7e2340f9cf0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:02.142750 8416 raft_consensus.cc:3060] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.143482 8546 tablet_bootstrap.cc:654] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.143852 8416 raft_consensus.cc:2468] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:02.144165 8220 leader_election.cc:304] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.144198 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2bf100407b034626b4a9f7e2340f9cf0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.144304 8146 raft_consensus.cc:3060] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.144359 9958 raft_consensus.cc:2804] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.144426 9958 raft_consensus.cc:697] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:02.144562 9958 consensus_queue.cc:237] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.145339 8146 raft_consensus.cc:2468] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:02.147768 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 626c2cc2d19047b794bf16c1b078941f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.147976 8551 tablet_replica.cc:333] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:02.148140 8551 raft_consensus.cc:2243] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.148264 8551 raft_consensus.cc:2272] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.148824 8551 ts_tablet_manager.cc:1916] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.151113 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 6ea1ef71a0974a5da443a61dc419c811 (DEFAULT_TABLE table=default.Table_8985e28c7f9d4c84aed8eed96c94a66c [id=78de796f78954453b000c8a90c7e9659]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.151412 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea1ef71a0974a5da443a61dc419c811. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.151587 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 6ea1ef71a0974a5da443a61dc419c811 (DEFAULT_TABLE table=default.Table_8985e28c7f9d4c84aed8eed96c94a66c [id=78de796f78954453b000c8a90c7e9659]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.151826 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea1ef71a0974a5da443a61dc419c811. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.152355 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 6ea1ef71a0974a5da443a61dc419c811 (DEFAULT_TABLE table=default.Table_8985e28c7f9d4c84aed8eed96c94a66c [id=78de796f78954453b000c8a90c7e9659]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.152742 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea1ef71a0974a5da443a61dc419c811. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.154882 8551 ts_tablet_manager.cc:1929] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.155033 8551 log.cc:1199] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/626c2cc2d19047b794bf16c1b078941f
I20251028 03:00:02.153913 9884 raft_consensus.cc:493] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.155411 8551 ts_tablet_manager.cc:1950] T 626c2cc2d19047b794bf16c1b078941f P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:02.155360 9884 raft_consensus.cc:515] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.155736 9884 leader_election.cc:290] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:02.155736 8332 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
W20251028 03:00:02.156692 8220 leader_election.cc:343] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 626c2cc2d19047b794bf16c1b078941f
W20251028 03:00:02.156787 8218 leader_election.cc:343] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 626c2cc2d19047b794bf16c1b078941f
I20251028 03:00:02.156867 8218 leader_election.cc:304] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:02.157256 9884 raft_consensus.cc:2749] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.157696 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 4b3047edf5a44514a23b689ee80ff1b4 (DEFAULT_TABLE table=default.Table_ba3d1f9ba1214c1d8ef3d9f737e91d6e [id=95158eff2f57463a8f0fda55abe6409a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.157866 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 4b3047edf5a44514a23b689ee80ff1b4 (DEFAULT_TABLE table=default.Table_ba3d1f9ba1214c1d8ef3d9f737e91d6e [id=95158eff2f57463a8f0fda55abe6409a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.157956 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b3047edf5a44514a23b689ee80ff1b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.158339 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 4b3047edf5a44514a23b689ee80ff1b4 (DEFAULT_TABLE table=default.Table_ba3d1f9ba1214c1d8ef3d9f737e91d6e [id=95158eff2f57463a8f0fda55abe6409a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.158597 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b3047edf5a44514a23b689ee80ff1b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.159529 8545 tablet_bootstrap.cc:492] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.159614 8545 ts_tablet_manager.cc:1403] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.001s
I20251028 03:00:02.159849 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9ad86438692e44b4915462c39c2c911d
I20251028 03:00:02.160156 8545 raft_consensus.cc:359] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.160256 8545 raft_consensus.cc:385] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.160285 8545 raft_consensus.cc:740] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.160595 8545 consensus_queue.cc:260] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.161226 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 8665d2e933fa423d81a1b32f2ce8629b (table default.Table_139255bc4fb74fae8fb86563b54826bd [id=dc3721126b09466ab5efdb8bd4b46d89]). Expected version 1 got 0
I20251028 03:00:02.161351 7965 catalog_manager.cc:5649] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.161523 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet ba40ba2c066e4c94885a595fd1c4445e (table default.Table_b2cfdddecdf14efcb61d58a4646b792f [id=25d0c2f2faef473b94ce2621281c08b0]). Expected version 2 got 0
I20251028 03:00:02.161875 8545 ts_tablet_manager.cc:1434] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20251028 03:00:02.162052 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 67f3b694dcc5465ca2c8c5535e7efea6 consensus state has no leader
I20251028 03:00:02.162290 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_642de38725d443f7861a2230d095c55c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:02.162528 8545 tablet_bootstrap.cc:492] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 03:00:02.162632 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_642de38725d443f7861a2230d095c55c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.163578 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3da0f96b500a4ff7b76f7aaa052a52a0" } new_table_name: "default.Table_74c1ac81e5664350bfef1e9e0161543a" modify_external_catalogs: true
I20251028 03:00:02.164582 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 626c2cc2d19047b794bf16c1b078941f (table default.Table_797c6e55c14f415589c745d9bedfcb07 [id=81dbfb2d58ab45958553cf73e259cdc0]) successfully deleted
I20251028 03:00:02.165568 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 9ad86438692e44b4915462c39c2c911d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.165935 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 9ad86438692e44b4915462c39c2c911d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.166131 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 9ad86438692e44b4915462c39c2c911d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.166450 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b3047edf5a44514a23b689ee80ff1b4. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:02.166728 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9ad86438692e44b4915462c39c2c911d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ad86438692e44b4915462c39c2c911d already in progress: creating tablet
W20251028 03:00:02.166911 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 9ad86438692e44b4915462c39c2c911d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ad86438692e44b4915462c39c2c911d already in progress: creating tablet
I20251028 03:00:02.166958 8545 tablet_bootstrap.cc:654] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:02.169637 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 9ad86438692e44b4915462c39c2c911d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ad86438692e44b4915462c39c2c911d already in progress: creating tablet
I20251028 03:00:02.175731 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 16424918ccc94191bb1efacdef642ca7 (DEFAULT_TABLE table=default.Table_25b2475406494c06be072da724e5351f [id=151912fc7db344d981223b0c7fae091e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.175858 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 626c2cc2d19047b794bf16c1b078941f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.176071 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16424918ccc94191bb1efacdef642ca7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.176076 8552 tablet_replica.cc:333] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:02.176227 8552 raft_consensus.cc:2243] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.176362 8552 raft_consensus.cc:2272] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.177028 8552 ts_tablet_manager.cc:1916] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.179885 8552 ts_tablet_manager.cc:1929] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.179971 8552 log.cc:1199] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/626c2cc2d19047b794bf16c1b078941f
I20251028 03:00:02.180207 8552 ts_tablet_manager.cc:1950] T 626c2cc2d19047b794bf16c1b078941f P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:02.181089 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 626c2cc2d19047b794bf16c1b078941f (table default.Table_797c6e55c14f415589c745d9bedfcb07 [id=81dbfb2d58ab45958553cf73e259cdc0]) successfully deleted
I20251028 03:00:02.182008 8545 tablet_bootstrap.cc:492] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.182103 8545 ts_tablet_manager.cc:1403] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251028 03:00:02.182585 8545 raft_consensus.cc:359] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.182690 8545 raft_consensus.cc:385] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.182721 8545 raft_consensus.cc:740] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.182916 8545 consensus_queue.cc:260] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.183370 8545 ts_tablet_manager.cc:1434] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.183560 8545 tablet_bootstrap.cc:492] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.185191 8545 tablet_bootstrap.cc:654] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.187214 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 16424918ccc94191bb1efacdef642ca7 (DEFAULT_TABLE table=default.Table_25b2475406494c06be072da724e5351f [id=151912fc7db344d981223b0c7fae091e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.187525 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16424918ccc94191bb1efacdef642ca7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.190371 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 16424918ccc94191bb1efacdef642ca7 (DEFAULT_TABLE table=default.Table_25b2475406494c06be072da724e5351f [id=151912fc7db344d981223b0c7fae091e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.190630 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16424918ccc94191bb1efacdef642ca7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.192629 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet d802f3cffe6a4e0cabbda0fb7fe495bc
I20251028 03:00:02.193066 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f96ae4167a29448e87878c857edcf307"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.193447 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f96ae4167a29448e87878c857edcf307 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.195587 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet d802f3cffe6a4e0cabbda0fb7fe495bc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:02.196583 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet d802f3cffe6a4e0cabbda0fb7fe495bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d802f3cffe6a4e0cabbda0fb7fe495bc already in progress: creating tablet
I20251028 03:00:02.197933 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet d802f3cffe6a4e0cabbda0fb7fe495bc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.198086 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet d802f3cffe6a4e0cabbda0fb7fe495bc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:02.198438 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet d802f3cffe6a4e0cabbda0fb7fe495bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d802f3cffe6a4e0cabbda0fb7fe495bc already in progress: creating tablet
I20251028 03:00:02.199361 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 1799f1947b08466395e588ff61492bf4 (DEFAULT_TABLE table=default.Table_4d5a8f2489b24f4eb6173b10a9328e41 [id=ed4b0f1e6893436d84901a04ccf93e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.199648 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1799f1947b08466395e588ff61492bf4. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:02.199649 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet d802f3cffe6a4e0cabbda0fb7fe495bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d802f3cffe6a4e0cabbda0fb7fe495bc already in progress: creating tablet
I20251028 03:00:02.200416 7961 catalog_manager.cc:5649] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.206657 8545 tablet_bootstrap.cc:492] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.206768 8545 ts_tablet_manager.cc:1403] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20251028 03:00:02.207335 8545 raft_consensus.cc:359] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.207477 8545 raft_consensus.cc:385] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.207526 8545 raft_consensus.cc:740] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.207662 8545 consensus_queue.cc:260] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.207717 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 1799f1947b08466395e588ff61492bf4 (DEFAULT_TABLE table=default.Table_4d5a8f2489b24f4eb6173b10a9328e41 [id=ed4b0f1e6893436d84901a04ccf93e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.207958 8545 ts_tablet_manager.cc:1434] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.208041 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1799f1947b08466395e588ff61492bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.208766 8545 tablet_bootstrap.cc:492] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.211145 8546 tablet_bootstrap.cc:492] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.211362 8546 ts_tablet_manager.cc:1403] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.069s user 0.000s sys 0.002s
I20251028 03:00:02.212028 8420 raft_consensus.cc:1275] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:02.212028 8146 raft_consensus.cc:1275] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer ccbdf478f18948d19de96574a685bfda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:02.212422 9595 consensus_queue.cc:1048] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.212539 9958 consensus_queue.cc:1048] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.213271 8546 raft_consensus.cc:359] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.213446 8546 raft_consensus.cc:385] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.213528 8546 raft_consensus.cc:740] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.213722 8546 consensus_queue.cc:260] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.215164 8546 ts_tablet_manager.cc:1434] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
W20251028 03:00:02.203778 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a279d68990284332bae453e4d26f1036 consensus state has no leader
I20251028 03:00:02.216866 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 1799f1947b08466395e588ff61492bf4 (DEFAULT_TABLE table=default.Table_4d5a8f2489b24f4eb6173b10a9328e41 [id=ed4b0f1e6893436d84901a04ccf93e00]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.217176 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1799f1947b08466395e588ff61492bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.217671 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ce2d5682714a4f6382b57097402f7f55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.218122 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce2d5682714a4f6382b57097402f7f55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.216866 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 2bf100407b034626b4a9f7e2340f9cf0 (table default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262]). Expected version 2 got 0
I20251028 03:00:02.219347 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 6b4245ce009043948113047840e54bb6 (DEFAULT_TABLE table=default.Table_0dc0518b395a459abe39e458996330e3 [id=878bfdb97c8f41718cfa752a2c6250ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.219635 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b4245ce009043948113047840e54bb6. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:02.222329 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 751777e4aac9489183e021c2367d3122 consensus state has no leader
I20251028 03:00:02.224294 8546 tablet_bootstrap.cc:492] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.225598 8546 tablet_bootstrap.cc:654] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.227865 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_85b63c813dc94719b07f0e12854b164b" } new_table_name: "default.Table_09ee3d6a580f4a32bf75d0bf858e64c6" modify_external_catalogs: true
I20251028 03:00:02.231387 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 6b4245ce009043948113047840e54bb6 (DEFAULT_TABLE table=default.Table_0dc0518b395a459abe39e458996330e3 [id=878bfdb97c8f41718cfa752a2c6250ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.231699 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b4245ce009043948113047840e54bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.232052 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 6b4245ce009043948113047840e54bb6 (DEFAULT_TABLE table=default.Table_0dc0518b395a459abe39e458996330e3 [id=878bfdb97c8f41718cfa752a2c6250ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.232298 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b4245ce009043948113047840e54bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.234927 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ca6ca361bde14c8aa060ffa12c7bf452
I20251028 03:00:02.238121 10187 tablet.cc:1722] T 40a0431501d348b19ec89fe692741ce5 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 03:00:02.238253 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet ca6ca361bde14c8aa060ffa12c7bf452 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.238581 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet ca6ca361bde14c8aa060ffa12c7bf452 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.239212 10192 tablet.cc:1722] T 40a0431501d348b19ec89fe692741ce5 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20251028 03:00:02.239331 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ca6ca361bde14c8aa060ffa12c7bf452 because the tablet was not found. No further retry: Not found: Tablet not found: ca6ca361bde14c8aa060ffa12c7bf452
W20251028 03:00:02.239990 7955 catalog_manager.cc:4952] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ca6ca361bde14c8aa060ffa12c7bf452 because the tablet was not found. No further retry: Not found: Tablet not found: ca6ca361bde14c8aa060ffa12c7bf452
I20251028 03:00:02.240561 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c676d258d54548aa89e8cc87d30171a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.240964 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c676d258d54548aa89e8cc87d30171a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.235998 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet ca6ca361bde14c8aa060ffa12c7bf452 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:02.243376 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ca6ca361bde14c8aa060ffa12c7bf452 because the tablet was not found. No further retry: Not found: Tablet not found: ca6ca361bde14c8aa060ffa12c7bf452
I20251028 03:00:02.244619 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 72cdf48bee6649c5a3144876265dccfe (DEFAULT_TABLE table=default.Table_5eb75de4388641e4bffe66784f304fcc [id=639b9aeb45ea442480663cabd64de9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.244520 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 72cdf48bee6649c5a3144876265dccfe (DEFAULT_TABLE table=default.Table_5eb75de4388641e4bffe66784f304fcc [id=639b9aeb45ea442480663cabd64de9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.244928 8544 tablet_bootstrap.cc:492] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:02.245025 8544 ts_tablet_manager.cc:1403] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.171s user 0.001s sys 0.001s
I20251028 03:00:02.245257 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72cdf48bee6649c5a3144876265dccfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.245500 8544 raft_consensus.cc:359] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.245608 8544 raft_consensus.cc:385] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.245652 8544 raft_consensus.cc:740] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.245783 8544 consensus_queue.cc:260] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.246115 8544 ts_tablet_manager.cc:1434] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.246297 8544 tablet_bootstrap.cc:492] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:02.252370 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72cdf48bee6649c5a3144876265dccfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.256784 8544 tablet_bootstrap.cc:654] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.258920 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 72cdf48bee6649c5a3144876265dccfe (DEFAULT_TABLE table=default.Table_5eb75de4388641e4bffe66784f304fcc [id=639b9aeb45ea442480663cabd64de9ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.259009 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f88362a62f9d434b9bbf51c049572ae1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:02.259210 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72cdf48bee6649c5a3144876265dccfe. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:02.259421 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f88362a62f9d434b9bbf51c049572ae1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.261451 8122 tablet_service.cc:1505] Processing CreateTablet for tablet ca2d2049891643cdb12e6e6a507cb493 (DEFAULT_TABLE table=default.Table_0a1e2cd2aace496790ca53a8bdfbea25 [id=cf6c49c2909c46ce97c6f51a2e7f5300]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.261935 8260 tablet_service.cc:1505] Processing CreateTablet for tablet ca2d2049891643cdb12e6e6a507cb493 (DEFAULT_TABLE table=default.Table_0a1e2cd2aace496790ca53a8bdfbea25 [id=cf6c49c2909c46ce97c6f51a2e7f5300]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.262492 8546 tablet_bootstrap.cc:492] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.262645 8546 ts_tablet_manager.cc:1403] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.002s
I20251028 03:00:02.263312 8546 raft_consensus.cc:359] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.263500 8546 raft_consensus.cc:385] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.263586 8546 raft_consensus.cc:740] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.263793 8546 consensus_queue.cc:260] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.264165 8546 ts_tablet_manager.cc:1434] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.264425 8546 tablet_bootstrap.cc:492] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.266441 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet e346247505904e819c837c019bc48806 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:02.264725 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2d2049891643cdb12e6e6a507cb493. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.269274 8546 tablet_bootstrap.cc:654] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.276168 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2d2049891643cdb12e6e6a507cb493. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.277486 9840 raft_consensus.cc:493] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.277595 9840 raft_consensus.cc:515] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.277639 7968 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 40a0431501d348b19ec89fe692741ce5 (table default.Table_a18c2020018b46ad99fea42906a9e93d [id=c103fff3458d4d08affefb7c38d35e52]). Expected version 3 got 0
I20251028 03:00:02.277997 9840 leader_election.cc:290] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.278584 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a5594c4b37154c58a5734fadcaf00972" } modify_external_catalogs: true
I20251028 03:00:02.278707 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a5594c4b37154c58a5734fadcaf00972" } modify_external_catalogs: true
I20251028 03:00:02.279237 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9a28ff8c0ee46cb89ba7e28de2cce45" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.279390 8143 raft_consensus.cc:2468] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.279660 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9a28ff8c0ee46cb89ba7e28de2cce45" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.279794 8282 raft_consensus.cc:2468] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.280520 8354 leader_election.cc:304] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:02.280678 9840 raft_consensus.cc:2804] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.280723 9840 raft_consensus.cc:493] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.280761 9840 raft_consensus.cc:3060] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.281512 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d0b0de6991ea465f9b83903824f98ad9" } modify_external_catalogs: true
I20251028 03:00:02.281683 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d0b0de6991ea465f9b83903824f98ad9" } modify_external_catalogs: true
I20251028 03:00:02.281850 9840 raft_consensus.cc:515] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.282215 9840 leader_election.cc:290] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.282783 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9a28ff8c0ee46cb89ba7e28de2cce45" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:02.282924 8282 raft_consensus.cc:3060] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.282938 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9a28ff8c0ee46cb89ba7e28de2cce45" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.283056 8143 raft_consensus.cc:3060] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.284036 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet 9621e42e6a324ecda50fe83c0c53119c with delete_type TABLET_DATA_DELETED (replaced by tablet bedb4fd3b46f4453b194cfaea0c3c701) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.284108 8143 raft_consensus.cc:2468] T e9a28ff8c0ee46cb89ba7e28de2cce45 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.284116 8282 raft_consensus.cc:2468] T e9a28ff8c0ee46cb89ba7e28de2cce45 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.284195 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ff112c66f1a4cf9b232e6bb665755f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.284379 8356 leader_election.cc:304] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.284473 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bc54972260747f3af0c18d05c6a3442 with delete_type TABLET_DATA_DELETED (replaced by tablet 9d438110cfec42a991b6c40b0881cc62) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.284390 8551 tablet_replica.cc:333] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:02.284701 8551 raft_consensus.cc:2243] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.284807 8551 raft_consensus.cc:2272] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.285195 9840 raft_consensus.cc:2804] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.285279 9840 raft_consensus.cc:697] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:02.285356 8551 ts_tablet_manager.cc:1916] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.285401 9840 consensus_queue.cc:237] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.286201 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.287216 8550 tablet_replica.cc:333] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:02.287403 8550 raft_consensus.cc:2243] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.287516 8550 raft_consensus.cc:2272] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.287931 8387 tablet_service.cc:1505] Processing CreateTablet for tablet ca2d2049891643cdb12e6e6a507cb493 (DEFAULT_TABLE table=default.Table_0a1e2cd2aace496790ca53a8bdfbea25 [id=cf6c49c2909c46ce97c6f51a2e7f5300]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.288435 8550 ts_tablet_manager.cc:1916] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.288448 8551 ts_tablet_manager.cc:1929] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.288453 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2d2049891643cdb12e6e6a507cb493. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.288540 8551 log.cc:1199] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9621e42e6a324ecda50fe83c0c53119c
I20251028 03:00:02.288759 8551 ts_tablet_manager.cc:1950] T 9621e42e6a324ecda50fe83c0c53119c P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:02.289505 8392 tablet_service.cc:1505] Processing CreateTablet for tablet a5fa3e89d7994f35bfbf55378101a0d9 (DEFAULT_TABLE table=default.Table_a56dbd80138241e084ed7c36f85873c7 [id=0a52d2eafc5246d89202485e578ca4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.289767 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5fa3e89d7994f35bfbf55378101a0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.290759 8550 ts_tablet_manager.cc:1929] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.290827 8550 log.cc:1199] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/6599ec60c3004e349e7e576e4d13c4fe
I20251028 03:00:02.291036 8550 ts_tablet_manager.cc:1950] T 6599ec60c3004e349e7e576e4d13c4fe P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:02.292057 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 6599ec60c3004e349e7e576e4d13c4fe (table default.Table_4f4721dca71f432b82e320f8fa9b0cf5 [id=bc3f005bc9424552831e2ecca5d049a7]) successfully deleted
I20251028 03:00:02.292224 8551 tablet_replica.cc:333] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:02.292227 9190 raft_consensus.cc:493] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.292464 9190 raft_consensus.cc:515] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.292824 9190 leader_election.cc:290] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.293315 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f422030a6a4042009efc7fcfb21a6bca" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.293483 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f422030a6a4042009efc7fcfb21a6bca" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.293602 8143 raft_consensus.cc:2468] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
W20251028 03:00:02.294833 8356 leader_election.cc:343] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.295011 7966 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet aa36c6870c1b4a459f0a41c84855a76e (table default.Table_d51065c1d57c4857bc5765d1baf9da06 [id=3815e9736bd34d668ec7f7e113e11e6f]). Expected version 1 got 0
I20251028 03:00:02.295089 8354 leader_election.cc:304] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.295429 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9621e42e6a324ecda50fe83c0c53119c (table default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]) successfully deleted
I20251028 03:00:02.295576 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a1f2e79fb7f743889dce73e1979b2832"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.295965 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1f2e79fb7f743889dce73e1979b2832 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.297010 9190 raft_consensus.cc:2804] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.297070 9190 raft_consensus.cc:493] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.297111 9190 raft_consensus.cc:3060] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.292389 8551 raft_consensus.cc:2243] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.297416 8551 raft_consensus.cc:2272] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.298012 8551 ts_tablet_manager.cc:1916] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.298175 9190 raft_consensus.cc:515] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.298557 9190 leader_election.cc:290] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.298924 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f422030a6a4042009efc7fcfb21a6bca" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:02.299286 8356 leader_election.cc:343] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.299649 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f422030a6a4042009efc7fcfb21a6bca" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.299772 8143 raft_consensus.cc:3060] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.300654 8143 raft_consensus.cc:2468] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.300925 8354 leader_election.cc:304] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.301294 9190 raft_consensus.cc:2804] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.301373 9190 raft_consensus.cc:697] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:02.301488 9190 consensus_queue.cc:237] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.302330 8551 ts_tablet_manager.cc:1929] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.302412 8551 log.cc:1199] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2ff112c66f1a4cf9b232e6bb665755f9
I20251028 03:00:02.302726 8551 ts_tablet_manager.cc:1950] T 2ff112c66f1a4cf9b232e6bb665755f9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:02.303715 8551 tablet_replica.cc:333] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:02.304077 8551 raft_consensus.cc:2243] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.304301 8551 raft_consensus.cc:2272] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.305111 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2ff112c66f1a4cf9b232e6bb665755f9 (table default.Table_9f07958f85cf4923a67bf8ac1ad1c722 [id=26d281a132204bc288eda2ac08794956]) successfully deleted
I20251028 03:00:02.305405 8112 tablet_service.cc:1505] Processing CreateTablet for tablet a5fa3e89d7994f35bfbf55378101a0d9 (DEFAULT_TABLE table=default.Table_a56dbd80138241e084ed7c36f85873c7 [id=0a52d2eafc5246d89202485e578ca4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.305624 8260 tablet_service.cc:1505] Processing CreateTablet for tablet a5fa3e89d7994f35bfbf55378101a0d9 (DEFAULT_TABLE table=default.Table_a56dbd80138241e084ed7c36f85873c7 [id=0a52d2eafc5246d89202485e578ca4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.305719 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5fa3e89d7994f35bfbf55378101a0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.305903 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5fa3e89d7994f35bfbf55378101a0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.309569 8112 tablet_service.cc:1505] Processing CreateTablet for tablet 899cdb5e4671477db1e850244b7952dd (DEFAULT_TABLE table=default.Table_47b181d6668e4b35972c604a53761997 [id=ccebe54daad3442fa04dfdf918b5186e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.309808 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 899cdb5e4671477db1e850244b7952dd (DEFAULT_TABLE table=default.Table_47b181d6668e4b35972c604a53761997 [id=ccebe54daad3442fa04dfdf918b5186e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.309913 8392 tablet_service.cc:1505] Processing CreateTablet for tablet 899cdb5e4671477db1e850244b7952dd (DEFAULT_TABLE table=default.Table_47b181d6668e4b35972c604a53761997 [id=ccebe54daad3442fa04dfdf918b5186e]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.309376 8551 ts_tablet_manager.cc:1916] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.310017 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 899cdb5e4671477db1e850244b7952dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.310050 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 899cdb5e4671477db1e850244b7952dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.310974 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 899cdb5e4671477db1e850244b7952dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.311089 8468 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
W20251028 03:00:02.315477 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 517df20e0f3245cfb1eaf1eb52d3c6a8 consensus state has no leader
I20251028 03:00:02.317696 9909 tablet.cc:1722] T 40a0431501d348b19ec89fe692741ce5 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 03:00:02.324741 7960 catalog_manager.cc:5649] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.325037 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (table default.Table_93b5aac358764b869e8589ccc023ea0e [id=674f76f9afed4f3f9e2c165e2479d938]). Expected version 1 got 0
I20251028 03:00:02.330902 9190 raft_consensus.cc:493] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.331028 9190 raft_consensus.cc:515] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.331463 9190 leader_election.cc:290] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.332357 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 9ea5e82c09354ea88fd67902625e1599 (DEFAULT_TABLE table=default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.333108 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a81841c22ae49c0a17931d90407accc" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.333240 8273 raft_consensus.cc:2468] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.333564 8356 leader_election.cc:304] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.334753 9840 raft_consensus.cc:2804] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.334817 9840 raft_consensus.cc:493] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.334859 9840 raft_consensus.cc:3060] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.336088 10192 tablet.cc:1722] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:02.336603 9840 raft_consensus.cc:515] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.336951 9840 leader_election.cc:290] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.338213 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a81841c22ae49c0a17931d90407accc" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:02.338331 8285 raft_consensus.cc:3060] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.338896 10187 tablet.cc:1722] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:02.339366 8285 raft_consensus.cc:2468] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.341106 8356 leader_election.cc:304] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.341607 9840 raft_consensus.cc:2804] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.341728 9840 raft_consensus.cc:697] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:02.342279 9840 consensus_queue.cc:237] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.345650 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet df10bbbd757a4da8905718b61cbb10c8 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:02.345624 8551 ts_tablet_manager.cc:1929] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.346294 8551 log.cc:1199] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/6bc54972260747f3af0c18d05c6a3442
I20251028 03:00:02.349128 8551 ts_tablet_manager.cc:1950] T 6bc54972260747f3af0c18d05c6a3442 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:02.334259 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a81841c22ae49c0a17931d90407accc" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.350930 8146 raft_consensus.cc:2468] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.342197 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9a81841c22ae49c0a17931d90407accc" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.351269 8144 raft_consensus.cc:3060] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.352371 8144 raft_consensus.cc:2468] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.342317 9909 tablet.cc:1722] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:02.353864 8545 tablet_bootstrap.cc:654] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.357515 8545 tablet_bootstrap.cc:492] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.357636 8545 ts_tablet_manager.cc:1403] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.149s user 0.003s sys 0.000s
I20251028 03:00:02.358146 8545 raft_consensus.cc:359] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.358260 8545 raft_consensus.cc:385] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.358255 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 6bc54972260747f3af0c18d05c6a3442 (table default.Table_4fe6d656356041dcaad42a9fd6f026d5 [id=f8375f32144c4541bea5dbcd51ab256e]) successfully deleted
I20251028 03:00:02.358294 8545 raft_consensus.cc:740] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.358412 8545 consensus_queue.cc:260] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.346767 8195 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:02.358692 8545 ts_tablet_manager.cc:1434] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.358878 8545 tablet_bootstrap.cc:492] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.360867 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 9b8fe701ffd44cffa62ec55be9edfe31 (DEFAULT_TABLE table=default.Table_c14c90440d424269b7a05b90f49e57e6 [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.361462 8392 tablet_service.cc:1505] Processing CreateTablet for tablet 9ea5e82c09354ea88fd67902625e1599 (DEFAULT_TABLE table=default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.361543 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 9b8fe701ffd44cffa62ec55be9edfe31 (DEFAULT_TABLE table=default.Table_c14c90440d424269b7a05b90f49e57e6 [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.361733 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea5e82c09354ea88fd67902625e1599. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.361827 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b8fe701ffd44cffa62ec55be9edfe31. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.363066 8544 tablet_bootstrap.cc:492] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:02.363168 8544 ts_tablet_manager.cc:1403] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.117s user 0.002s sys 0.001s
I20251028 03:00:02.363744 8544 raft_consensus.cc:359] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.363914 8544 raft_consensus.cc:385] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.363950 8544 raft_consensus.cc:740] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.364084 8544 consensus_queue.cc:260] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.364217 8545 tablet_bootstrap.cc:654] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.364387 8544 ts_tablet_manager.cc:1434] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 03:00:02.366272 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c2771559bf9e405cb2b3d14ea5f64966"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.366690 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2771559bf9e405cb2b3d14ea5f64966 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.371544 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 9ea5e82c09354ea88fd67902625e1599 (DEFAULT_TABLE table=default.Table_bb369e38928a4e20af6d144f13efd7ac [id=2817a81686f2452db800a03983aa5831]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.371922 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea5e82c09354ea88fd67902625e1599. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.376065 8545 tablet_bootstrap.cc:492] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.376178 8545 ts_tablet_manager.cc:1403] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 03:00:02.376694 8545 raft_consensus.cc:359] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.376811 8545 raft_consensus.cc:385] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.376845 8545 raft_consensus.cc:740] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.376967 8545 consensus_queue.cc:260] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.377285 8545 ts_tablet_manager.cc:1434] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.377460 8545 tablet_bootstrap.cc:492] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.377887 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 4df3fcb884174a1580c8f61663380673 (DEFAULT_TABLE table=default.Table_ec5ff388e162425e96cc5c12f2e8f804 [id=298f07dc9ad2434ea639510d7788641f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.378378 8545 tablet_bootstrap.cc:654] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.379259 8544 tablet_bootstrap.cc:492] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:02.380940 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 4df3fcb884174a1580c8f61663380673 (DEFAULT_TABLE table=default.Table_ec5ff388e162425e96cc5c12f2e8f804 [id=298f07dc9ad2434ea639510d7788641f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.381215 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df3fcb884174a1580c8f61663380673. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.381403 8392 tablet_service.cc:1505] Processing CreateTablet for tablet 9b8fe701ffd44cffa62ec55be9edfe31 (DEFAULT_TABLE table=default.Table_c14c90440d424269b7a05b90f49e57e6 [id=5f82b9596e5842deb4d45e19c91629bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.381605 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b8fe701ffd44cffa62ec55be9edfe31. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.383972 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 4df3fcb884174a1580c8f61663380673 (DEFAULT_TABLE table=default.Table_ec5ff388e162425e96cc5c12f2e8f804 [id=298f07dc9ad2434ea639510d7788641f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.384315 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df3fcb884174a1580c8f61663380673. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.387204 9884 raft_consensus.cc:493] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.387315 9884 raft_consensus.cc:515] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.387784 9884 leader_election.cc:290] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:02.388190 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba40ba2c066e4c94885a595fd1c4445e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:02.389067 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b8fe701ffd44cffa62ec55be9edfe31. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.393194 8545 tablet_bootstrap.cc:492] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.393302 8545 ts_tablet_manager.cc:1403] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 03:00:02.394022 8545 raft_consensus.cc:359] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.394143 8545 raft_consensus.cc:385] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.394176 8545 raft_consensus.cc:740] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.394692 8545 consensus_queue.cc:260] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.395105 8545 ts_tablet_manager.cc:1434] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:02.395378 8545 tablet_bootstrap.cc:492] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.396462 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba40ba2c066e4c94885a595fd1c4445e" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.396612 8143 raft_consensus.cc:2393] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 03:00:02.397923 8545 tablet_bootstrap.cc:654] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.399192 8545 tablet_bootstrap.cc:492] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.399276 8545 ts_tablet_manager.cc:1403] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 03:00:02.399946 8545 raft_consensus.cc:359] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.400067 8545 raft_consensus.cc:385] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.400100 8545 raft_consensus.cc:740] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.400213 8545 consensus_queue.cc:260] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.400507 8545 ts_tablet_manager.cc:1434] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.401696 8218 leader_election.cc:304] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:02.402076 9884 raft_consensus.cc:3060] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.403208 9884 raft_consensus.cc:2749] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.405603 8544 tablet_bootstrap.cc:654] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.406898 8544 tablet_bootstrap.cc:492] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:02.407079 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea5e82c09354ea88fd67902625e1599. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.407120 8544 ts_tablet_manager.cc:1403] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.002s
I20251028 03:00:02.407598 8544 raft_consensus.cc:359] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.407712 8544 raft_consensus.cc:385] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.407740 8544 raft_consensus.cc:740] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.407861 8544 consensus_queue.cc:260] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.407938 9788 raft_consensus.cc:493] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.408011 9788 raft_consensus.cc:515] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.408373 9788 leader_election.cc:290] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.408403 8544 ts_tablet_manager.cc:1434] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.408582 8544 tablet_bootstrap.cc:492] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:02.409492 8544 tablet_bootstrap.cc:654] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:02.409687 7954 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 3a5415a4246c47ca896751f2fd68af71 (table default.Table_4bafc1e32c454d84b181dadf3727efcc [id=077d81b3977448b2ae212436d5014339], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3a5415a4246c47ca896751f2fd68af71 consensus state has no leader
I20251028 03:00:02.409724 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c0081e0ce4d4e499c37f217fe62c546" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:02.410135 8083 leader_election.cc:343] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.411437 8545 tablet_bootstrap.cc:492] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.411424 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c0081e0ce4d4e499c37f217fe62c546" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:02.411787 8083 leader_election.cc:343] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.411868 8083 leader_election.cc:304] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:02.412249 9788 raft_consensus.cc:2749] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.412395 10242 raft_consensus.cc:493] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.412490 10242 raft_consensus.cc:515] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.412937 10242 leader_election.cc:290] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.412408 8545 tablet_bootstrap.cc:654] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.413952 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d609dc2fea4c79adf0953dc98ad0af" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:02.414299 8356 leader_election.cc:343] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.415257 8545 tablet_bootstrap.cc:492] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.415452 8545 ts_tablet_manager.cc:1403] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20251028 03:00:02.415932 8545 raft_consensus.cc:359] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.416035 8545 raft_consensus.cc:385] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.416066 8545 raft_consensus.cc:740] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.416186 8545 consensus_queue.cc:260] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.417631 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d609dc2fea4c79adf0953dc98ad0af" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.417768 8144 raft_consensus.cc:2468] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.417800 8545 ts_tablet_manager.cc:1434] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251028 03:00:02.418123 8545 tablet_bootstrap.cc:492] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.418236 9788 raft_consensus.cc:493] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.418315 9788 raft_consensus.cc:515] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.418670 9788 leader_election.cc:290] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:02.419247 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5c1994484abf4b6fbbc3f47527558cdb" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.419690 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5c1994484abf4b6fbbc3f47527558cdb" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:02.419847 8420 raft_consensus.cc:2468] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
W20251028 03:00:02.419864 8083 leader_election.cc:343] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.420784 8083 leader_election.cc:304] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.421051 9788 raft_consensus.cc:2804] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.421115 9788 raft_consensus.cc:493] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.421160 9788 raft_consensus.cc:3060] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.421881 8545 tablet_bootstrap.cc:654] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.422259 9788 raft_consensus.cc:515] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.422572 9788 leader_election.cc:290] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:02.423368 8285 raft_consensus.cc:1275] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:02.423708 8544 tablet_bootstrap.cc:492] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:02.423791 8544 ts_tablet_manager.cc:1403] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.001s
I20251028 03:00:02.423848 9840 consensus_queue.cc:1048] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.424206 8544 raft_consensus.cc:359] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.424296 8544 raft_consensus.cc:385] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.424322 8544 raft_consensus.cc:740] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.424419 8544 consensus_queue.cc:260] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.424654 8544 ts_tablet_manager.cc:1434] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:02.429587 10192 tablet.cc:1722] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:02.431795 10187 tablet.cc:1722] T ba40ba2c066e4c94885a595fd1c4445e P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:02.432595 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5c1994484abf4b6fbbc3f47527558cdb" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:02.432719 8420 raft_consensus.cc:3060] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.433732 8420 raft_consensus.cc:2468] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:02.433861 9190 raft_consensus.cc:493] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.433946 9190 raft_consensus.cc:515] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.434067 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5c1994484abf4b6fbbc3f47527558cdb" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:02.434285 9190 leader_election.cc:290] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.434762 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "338cf9d02142436f97ccd44313cc8199" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:02.435268 8356 leader_election.cc:343] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.435487 8083 leader_election.cc:304] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
W20251028 03:00:02.435671 8083 leader_election.cc:343] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.435798 10253 raft_consensus.cc:2804] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.435961 10253 raft_consensus.cc:697] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:02.436286 10253 consensus_queue.cc:237] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.437953 8332 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:02.438372 8545 tablet_bootstrap.cc:492] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.438467 8545 ts_tablet_manager.cc:1403] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.001s
I20251028 03:00:02.438987 8545 raft_consensus.cc:359] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.439097 8545 raft_consensus.cc:385] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.439136 8545 raft_consensus.cc:740] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.439277 8545 consensus_queue.cc:260] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.439604 8545 ts_tablet_manager.cc:1434] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.439806 8545 tablet_bootstrap.cc:492] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.440865 8545 tablet_bootstrap.cc:654] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.441926 8544 tablet_bootstrap.cc:492] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:02.442926 8544 tablet_bootstrap.cc:654] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.445562 8144 raft_consensus.cc:1275] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Refusing update from remote peer d7e4cf694f71438298dd32a346abcf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:02.445921 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "338cf9d02142436f97ccd44313cc8199" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.446033 8146 raft_consensus.cc:2468] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.446486 8354 leader_election.cc:304] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.446869 10242 consensus_queue.cc:1048] T ba40ba2c066e4c94885a595fd1c4445e P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.446877 9190 raft_consensus.cc:2804] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.447103 9190 raft_consensus.cc:493] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.447175 9190 raft_consensus.cc:3060] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.448665 8354 leader_election.cc:304] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.448907 10242 raft_consensus.cc:2804] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.448987 10242 raft_consensus.cc:493] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.449036 10242 raft_consensus.cc:3060] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.451668 9909 tablet.cc:1722] T ba40ba2c066e4c94885a595fd1c4445e P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:02.452679 8544 tablet_bootstrap.cc:492] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:02.452782 8544 ts_tablet_manager.cc:1403] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 03:00:02.453269 8544 raft_consensus.cc:359] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.453375 8544 raft_consensus.cc:385] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.453405 8544 raft_consensus.cc:740] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.453526 8544 consensus_queue.cc:260] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.453833 8544 ts_tablet_manager.cc:1434] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:02.454042 8544 tablet_bootstrap.cc:492] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:02.455114 8544 tablet_bootstrap.cc:654] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.455368 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4df3fcb884174a1580c8f61663380673. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.458707 8263 tablet_service.cc:1505] Processing CreateTablet for tablet e8048ecc19ea45af9abf964c84d3ebfa (DEFAULT_TABLE table=default.Table_6de616ea4b53462a81fe546b5cc0279c [id=329bf229edbc45d48c2ea0710fe216ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.458776 8260 tablet_service.cc:1505] Processing CreateTablet for tablet ca6ca361bde14c8aa060ffa12c7bf452 (DEFAULT_TABLE table=default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.459021 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8048ecc19ea45af9abf964c84d3ebfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.459183 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca6ca361bde14c8aa060ffa12c7bf452. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.459609 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 86cd4029953949f7994766653bf40b7c (DEFAULT_TABLE table=default.Table_b760e30f659c4e21b76a578756da6071 [id=f653d20f14874fb5a608d34babe35594]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.459823 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86cd4029953949f7994766653bf40b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.460754 8392 tablet_service.cc:1505] Processing CreateTablet for tablet ca6ca361bde14c8aa060ffa12c7bf452 (DEFAULT_TABLE table=default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.461015 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca6ca361bde14c8aa060ffa12c7bf452. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.461434 8387 tablet_service.cc:1505] Processing CreateTablet for tablet e8048ecc19ea45af9abf964c84d3ebfa (DEFAULT_TABLE table=default.Table_6de616ea4b53462a81fe546b5cc0279c [id=329bf229edbc45d48c2ea0710fe216ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.461661 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8048ecc19ea45af9abf964c84d3ebfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.461436 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 86cd4029953949f7994766653bf40b7c (DEFAULT_TABLE table=default.Table_b760e30f659c4e21b76a578756da6071 [id=f653d20f14874fb5a608d34babe35594]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.462083 8546 tablet_bootstrap.cc:492] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.462139 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86cd4029953949f7994766653bf40b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.462167 8546 ts_tablet_manager.cc:1403] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.198s user 0.002s sys 0.000s
I20251028 03:00:02.462637 8546 raft_consensus.cc:359] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.462743 8546 raft_consensus.cc:385] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.462788 8546 raft_consensus.cc:740] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.462913 8546 consensus_queue.cc:260] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.463240 8546 ts_tablet_manager.cc:1434] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.463464 8546 tablet_bootstrap.cc:492] T 01d609dc2fea4c79adf0953dc98ad0af P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.464344 8546 tablet_bootstrap.cc:654] T 01d609dc2fea4c79adf0953dc98ad0af P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.464339 8112 tablet_service.cc:1505] Processing CreateTablet for tablet e8048ecc19ea45af9abf964c84d3ebfa (DEFAULT_TABLE table=default.Table_6de616ea4b53462a81fe546b5cc0279c [id=329bf229edbc45d48c2ea0710fe216ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.464581 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8048ecc19ea45af9abf964c84d3ebfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.464927 8120 tablet_service.cc:1505] Processing CreateTablet for tablet ca6ca361bde14c8aa060ffa12c7bf452 (DEFAULT_TABLE table=default.Table_00e05a1a71944e26a7e93167c3fdce40 [id=623eeb8488074e6c96d45caf933c95ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.465152 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca6ca361bde14c8aa060ffa12c7bf452. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.465791 8546 tablet_bootstrap.cc:492] T 01d609dc2fea4c79adf0953dc98ad0af P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.465942 8546 ts_tablet_manager.cc:1403] T 01d609dc2fea4c79adf0953dc98ad0af P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 03:00:02.466689 8546 raft_consensus.cc:359] T 01d609dc2fea4c79adf0953dc98ad0af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.466871 8546 raft_consensus.cc:385] T 01d609dc2fea4c79adf0953dc98ad0af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.466950 8546 raft_consensus.cc:740] T 01d609dc2fea4c79adf0953dc98ad0af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.467146 8546 consensus_queue.cc:260] T 01d609dc2fea4c79adf0953dc98ad0af P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.467506 8546 ts_tablet_manager.cc:1434] T 01d609dc2fea4c79adf0953dc98ad0af P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.467815 8546 tablet_bootstrap.cc:492] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 03:00:02.467816 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f5e7e0614af04cc2a22130906d062a79 consensus state has no leader
I20251028 03:00:02.469012 8546 tablet_bootstrap.cc:654] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.470418 8546 tablet_bootstrap.cc:492] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.470507 8546 ts_tablet_manager.cc:1403] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 03:00:02.470957 8546 raft_consensus.cc:359] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.471051 8546 raft_consensus.cc:385] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.471084 8546 raft_consensus.cc:740] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.471199 8546 consensus_queue.cc:260] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.471714 8545 tablet_bootstrap.cc:492] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.471802 8545 ts_tablet_manager.cc:1403] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20251028 03:00:02.472297 8545 raft_consensus.cc:359] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.472397 8545 raft_consensus.cc:385] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.472430 8545 raft_consensus.cc:740] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.472543 8545 consensus_queue.cc:260] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.472811 8545 ts_tablet_manager.cc:1434] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.473006 8545 tablet_bootstrap.cc:492] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.474045 8545 tablet_bootstrap.cc:654] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.475808 10242 raft_consensus.cc:515] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.476224 10242 leader_election.cc:290] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.476646 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "338cf9d02142436f97ccd44313cc8199" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:02.476997 8356 leader_election.cc:343] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.477154 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet b5e87126939c4d559522e7738a81e1d2
I20251028 03:00:02.477535 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 5af63f43d2f64606ab1becacf75367c9 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:02.477774 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet b5e87126939c4d559522e7738a81e1d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.477923 9190 raft_consensus.cc:515] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.478279 9190 leader_election.cc:290] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:02.478296 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet b5e87126939c4d559522e7738a81e1d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b5e87126939c4d559522e7738a81e1d2 already in progress: creating tablet
I20251028 03:00:02.478565 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d609dc2fea4c79adf0953dc98ad0af" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:02.478669 8285 raft_consensus.cc:3060] T 01d609dc2fea4c79adf0953dc98ad0af P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.478849 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3f475aaefd244e529f54110a86eb240f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.479244 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f475aaefd244e529f54110a86eb240f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.479249 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet b5e87126939c4d559522e7738a81e1d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.479777 8285 raft_consensus.cc:2468] T 01d609dc2fea4c79adf0953dc98ad0af P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.480516 8356 leader_election.cc:304] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
W20251028 03:00:02.480824 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet b5e87126939c4d559522e7738a81e1d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b5e87126939c4d559522e7738a81e1d2 already in progress: creating tablet
I20251028 03:00:02.481040 9190 raft_consensus.cc:2804] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.481115 9190 raft_consensus.cc:697] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:02.481098 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b760e30f659c4e21b76a578756da6071" } modify_external_catalogs: true
I20251028 03:00:02.481213 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b760e30f659c4e21b76a578756da6071" } modify_external_catalogs: true
I20251028 03:00:02.481225 9190 consensus_queue.cc:237] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.482241 8546 ts_tablet_manager.cc:1434] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20251028 03:00:02.482463 8546 tablet_bootstrap.cc:492] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.483814 8546 tablet_bootstrap.cc:654] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.482169 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d51065c1d57c4857bc5765d1baf9da06" } new_table_name: "default.Table_86f07803f4c84e848af88e14238e1407" modify_external_catalogs: true
I20251028 03:00:02.486862 7962 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet df10bbbd757a4da8905718b61cbb10c8
I20251028 03:00:02.487128 7962 catalog_manager.cc:6719] ReplaceTablet: tablet df10bbbd757a4da8905718b61cbb10c8 deleted and replaced by tablet f9d1b4032a694a879b794216554f4413
I20251028 03:00:02.487566 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet df10bbbd757a4da8905718b61cbb10c8 with delete_type TABLET_DATA_DELETED (replaced by tablet f9d1b4032a694a879b794216554f4413) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.487854 8545 tablet_bootstrap.cc:492] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.487890 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet df10bbbd757a4da8905718b61cbb10c8 with delete_type TABLET_DATA_DELETED (replaced by tablet f9d1b4032a694a879b794216554f4413) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.487955 8545 ts_tablet_manager.cc:1403] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.001s
I20251028 03:00:02.488081 8551 tablet_replica.cc:333] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:02.488206 8551 raft_consensus.cc:2243] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.488313 8551 raft_consensus.cc:2272] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.488483 8552 tablet_replica.cc:333] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:02.488440 8545 raft_consensus.cc:359] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.488533 8545 raft_consensus.cc:385] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.488561 8545 raft_consensus.cc:740] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.488633 8552 raft_consensus.cc:2243] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:02.488672 8545 consensus_queue.cc:260] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.488862 8552 raft_consensus.cc:2272] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.488960 8545 ts_tablet_manager.cc:1434] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.489094 8545 tablet_bootstrap.cc:492] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.489228 8552 ts_tablet_manager.cc:1916] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.489995 8545 tablet_bootstrap.cc:654] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.488673 8551 ts_tablet_manager.cc:1916] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.493333 8551 ts_tablet_manager.cc:1929] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20251028 03:00:02.493395 8551 log.cc:1199] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/df10bbbd757a4da8905718b61cbb10c8
I20251028 03:00:02.493636 8551 ts_tablet_manager.cc:1950] T df10bbbd757a4da8905718b61cbb10c8 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:02.493803 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2b2a784556c14e418b568477963b0814"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.494184 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b2a784556c14e418b568477963b0814 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.494428 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet df10bbbd757a4da8905718b61cbb10c8 (table default.Table_927bd3d6ebc54f3aaa7c47e636452f05 [id=a6ae2be5375e4328848f28f43dfc49ab]) successfully deleted
I20251028 03:00:02.494979 7968 catalog_manager.cc:5931] Not sending DeleteTablet requests; no consensus state for tablet 33d081ead7624fcc86f757e9e43dc9ec
I20251028 03:00:02.495167 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_206be08f6668466db643e3bf2fff50ce" } modify_external_catalogs: true
I20251028 03:00:02.495255 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_206be08f6668466db643e3bf2fff50ce" } modify_external_catalogs: true
I20251028 03:00:02.496435 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 7e0e038d33bf45f693da251cd560ae2a from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:02.496595 8545 tablet_bootstrap.cc:492] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.496682 8545 ts_tablet_manager.cc:1403] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20251028 03:00:02.497346 8545 raft_consensus.cc:359] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.497460 8545 raft_consensus.cc:385] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.497500 8545 raft_consensus.cc:740] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.497654 8545 consensus_queue.cc:260] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.497964 8545 ts_tablet_manager.cc:1434] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.498231 8552 ts_tablet_manager.cc:1929] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20251028 03:00:02.498347 8552 log.cc:1199] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/df10bbbd757a4da8905718b61cbb10c8
I20251028 03:00:02.498430 8545 tablet_bootstrap.cc:492] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.498593 8468 heartbeater.cc:344] Connected to a master server at 127.7.154.190:39091
I20251028 03:00:02.498661 8552 ts_tablet_manager.cc:1950] T df10bbbd757a4da8905718b61cbb10c8 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:02.499349 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet df10bbbd757a4da8905718b61cbb10c8 (table default.Table_927bd3d6ebc54f3aaa7c47e636452f05 [id=a6ae2be5375e4328848f28f43dfc49ab]) successfully deleted
I20251028 03:00:02.500146 8545 tablet_bootstrap.cc:654] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.501266 8546 tablet_bootstrap.cc:492] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.501358 8546 ts_tablet_manager.cc:1403] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20251028 03:00:02.501825 8546 raft_consensus.cc:359] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.501926 8546 raft_consensus.cc:385] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.501960 8546 raft_consensus.cc:740] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.502076 8546 consensus_queue.cc:260] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.502357 8546 ts_tablet_manager.cc:1434] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.502540 8546 tablet_bootstrap.cc:492] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.508671 8545 tablet_bootstrap.cc:492] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.508770 8545 ts_tablet_manager.cc:1403] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.003s
I20251028 03:00:02.509255 8545 raft_consensus.cc:359] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.509371 8545 raft_consensus.cc:385] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.509413 8545 raft_consensus.cc:740] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.509536 8545 consensus_queue.cc:260] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.509871 8545 ts_tablet_manager.cc:1434] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:02.510115 8545 tablet_bootstrap.cc:492] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.510730 8546 tablet_bootstrap.cc:654] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.511111 8545 tablet_bootstrap.cc:654] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.512279 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet df10bbbd757a4da8905718b61cbb10c8 with delete_type TABLET_DATA_DELETED (replaced by tablet f9d1b4032a694a879b794216554f4413) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.512486 8550 tablet_replica.cc:333] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:02.512660 8550 raft_consensus.cc:2243] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.512784 8550 raft_consensus.cc:2272] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.513186 8550 ts_tablet_manager.cc:1916] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.515722 8550 ts_tablet_manager.cc:1929] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20251028 03:00:02.515796 8550 log.cc:1199] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/df10bbbd757a4da8905718b61cbb10c8
I20251028 03:00:02.516048 8550 ts_tablet_manager.cc:1950] T df10bbbd757a4da8905718b61cbb10c8 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:02.517001 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet df10bbbd757a4da8905718b61cbb10c8 (table default.Table_927bd3d6ebc54f3aaa7c47e636452f05 [id=a6ae2be5375e4328848f28f43dfc49ab]) successfully deleted
I20251028 03:00:02.517589 8545 tablet_bootstrap.cc:492] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.517681 8545 ts_tablet_manager.cc:1403] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 03:00:02.518177 8545 raft_consensus.cc:359] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.518288 8545 raft_consensus.cc:385] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.518317 8545 raft_consensus.cc:740] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.518430 8545 consensus_queue.cc:260] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.518733 8545 ts_tablet_manager.cc:1434] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.518905 8545 tablet_bootstrap.cc:492] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.520112 8545 tablet_bootstrap.cc:654] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.523762 10242 raft_consensus.cc:493] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.523860 10242 raft_consensus.cc:515] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.524241 10242 leader_election.cc:290] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.524597 8546 tablet_bootstrap.cc:492] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.524703 8546 ts_tablet_manager.cc:1403] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20251028 03:00:02.524822 8144 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "01d609dc2fea4c79adf0953dc98ad0af" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.524948 8144 raft_consensus.cc:3060] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.525275 8546 raft_consensus.cc:359] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.525372 8546 raft_consensus.cc:385] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.525398 8546 raft_consensus.cc:740] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.525496 8546 consensus_queue.cc:260] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.525804 8546 ts_tablet_manager.cc:1434] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.525918 8545 tablet_bootstrap.cc:492] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.526010 8545 ts_tablet_manager.cc:1403] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20251028 03:00:02.525996 8144 raft_consensus.cc:2468] T 01d609dc2fea4c79adf0953dc98ad0af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.526036 8546 tablet_bootstrap.cc:492] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.526504 8545 raft_consensus.cc:359] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.526625 8545 raft_consensus.cc:385] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.526667 8545 raft_consensus.cc:740] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.526786 8545 consensus_queue.cc:260] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.527145 8545 ts_tablet_manager.cc:1434] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:02.527186 8546 tablet_bootstrap.cc:654] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.527374 8545 tablet_bootstrap.cc:492] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.527405 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd8c04efd8c2456b9c7643641c7fbc0d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.527423 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "338cf9d02142436f97ccd44313cc8199" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.527506 8143 raft_consensus.cc:2468] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.527526 8146 raft_consensus.cc:3060] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.528563 8146 raft_consensus.cc:2468] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.528602 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd8c04efd8c2456b9c7643641c7fbc0d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.528697 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet b5e87126939c4d559522e7738a81e1d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.529104 8354 leader_election.cc:304] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:02.529420 9190 raft_consensus.cc:2804] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.529469 9190 raft_consensus.cc:493] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.529503 9190 raft_consensus.cc:3060] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.530507 9190 raft_consensus.cc:515] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.530869 9190 leader_election.cc:290] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.531092 9788 raft_consensus.cc:493] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.531179 9788 raft_consensus.cc:515] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.531548 9788 leader_election.cc:290] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:02.532559 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd8c04efd8c2456b9c7643641c7fbc0d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.532570 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd8c04efd8c2456b9c7643641c7fbc0d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:02.532663 8148 raft_consensus.cc:3060] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.532707 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 86cd4029953949f7994766653bf40b7c (DEFAULT_TABLE table=default.Table_b760e30f659c4e21b76a578756da6071 [id=f653d20f14874fb5a608d34babe35594]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 03:00:02.532994 8356 leader_election.cc:343] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.533253 8354 leader_election.cc:304] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.533663 8148 raft_consensus.cc:2468] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
W20251028 03:00:02.533730 8356 leader_election.cc:343] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.533967 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ad926e9ee0044c62a7abbfca959a4f56" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:02.534078 8420 raft_consensus.cc:2468] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:02.534482 8083 leader_election.cc:304] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:02.534657 9788 raft_consensus.cc:2804] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.534711 9788 raft_consensus.cc:493] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.534754 9788 raft_consensus.cc:3060] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.535727 9788 raft_consensus.cc:515] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.535812 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ad926e9ee0044c62a7abbfca959a4f56" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.536042 10242 raft_consensus.cc:2804] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.536109 9788 leader_election.cc:290] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:02.536130 10242 raft_consensus.cc:697] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:02.536271 10242 consensus_queue.cc:237] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.536362 8354 leader_election.cc:304] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.536716 10246 raft_consensus.cc:2804] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.536732 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ad926e9ee0044c62a7abbfca959a4f56" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:02.536823 10246 raft_consensus.cc:697] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:02.536859 8420 raft_consensus.cc:3060] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.536942 10246 consensus_queue.cc:237] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:02.537854 8083 leader_election.cc:343] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.537878 8420 raft_consensus.cc:2468] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:02.538210 8545 tablet_bootstrap.cc:654] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.538347 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ad926e9ee0044c62a7abbfca959a4f56" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:02.540278 8083 leader_election.cc:304] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
W20251028 03:00:02.540714 8083 leader_election.cc:343] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.540993 9788 raft_consensus.cc:2804] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.541074 9788 raft_consensus.cc:697] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:02.541190 9788 consensus_queue.cc:237] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 03:00:02.546859 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet b5e87126939c4d559522e7738a81e1d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b5e87126939c4d559522e7738a81e1d2 already in progress: creating tablet
I20251028 03:00:02.551120 9190 raft_consensus.cc:493] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.551267 9190 raft_consensus.cc:515] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.551708 9190 leader_election.cc:290] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.552165 9884 raft_consensus.cc:493] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.552217 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8665d2e933fa423d81a1b32f2ce8629b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.552259 9884 raft_consensus.cc:515] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.552322 8285 raft_consensus.cc:2468] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.552599 9884 leader_election.cc:290] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:02.552904 8356 leader_election.cc:304] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.553205 9190 raft_consensus.cc:2804] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.553267 9190 raft_consensus.cc:493] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.553318 9190 raft_consensus.cc:3060] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.553469 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f422030a6a4042009efc7fcfb21a6bca" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:02.554456 9190 raft_consensus.cc:515] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.554679 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f422030a6a4042009efc7fcfb21a6bca" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.554787 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8665d2e933fa423d81a1b32f2ce8629b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.554836 8142 raft_consensus.cc:2393] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 03:00:02.554924 8148 raft_consensus.cc:2468] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.555004 9190 leader_election.cc:290] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.555528 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8665d2e933fa423d81a1b32f2ce8629b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:02.555662 8285 raft_consensus.cc:3060] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.556017 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86cd4029953949f7994766653bf40b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.556332 8218 leader_election.cc:304] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:02.556540 9884 raft_consensus.cc:3060] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.556793 8285 raft_consensus.cc:2468] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.557071 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8665d2e933fa423d81a1b32f2ce8629b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.557171 8148 raft_consensus.cc:3060] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.557648 9884 raft_consensus.cc:2749] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.557991 8356 leader_election.cc:304] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.558224 8148 raft_consensus.cc:2468] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.560429 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 86cd4029953949f7994766653bf40b7c
I20251028 03:00:02.560876 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 04723e51ee66454182da19634c14ffb4 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:02.561049 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 86cd4029953949f7994766653bf40b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.561105 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 86cd4029953949f7994766653bf40b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.561203 9190 raft_consensus.cc:2804] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
W20251028 03:00:02.561529 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 86cd4029953949f7994766653bf40b7c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 86cd4029953949f7994766653bf40b7c already in progress: creating tablet
I20251028 03:00:02.561570 9190 raft_consensus.cc:697] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:02.561718 9190 consensus_queue.cc:237] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:02.563148 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 86cd4029953949f7994766653bf40b7c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 86cd4029953949f7994766653bf40b7c already in progress: creating tablet
I20251028 03:00:02.564051 10101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: Bootstrap replayed 1/1 log segments. Stats: ops{read=934 overwritten=0 applied=934 ignored=0} inserts{seen=746 ignored=0} mutations{seen=1118 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251028 03:00:02.564785 10101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: Bootstrap complete.
I20251028 03:00:02.568603 10101 raft_consensus.cc:359] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 03:00:02.569187 10101 raft_consensus.cc:740] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac9bf5a198d042698c9f030a59228865, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.569584 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5af63f43d2f64606ab1becacf75367c9
I20251028 03:00:02.569896 7965 catalog_manager.cc:6719] ReplaceTablet: tablet 5af63f43d2f64606ab1becacf75367c9 deleted and replaced by tablet 94a43fc2b1694823919e33c2dafa5037
I20251028 03:00:02.569832 10101 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 934, Last appended: 2.934, Last appended by leader: 934, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 03:00:02.570704 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 5af63f43d2f64606ab1becacf75367c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 94a43fc2b1694823919e33c2dafa5037) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.571790 10101 sys_catalog.cc:565] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: configured and running, proceeding with master startup.
W20251028 03:00:02.572168 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 5af63f43d2f64606ab1becacf75367c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5af63f43d2f64606ab1becacf75367c9 already in progress: creating tablet
I20251028 03:00:02.572594 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 5af63f43d2f64606ab1becacf75367c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 94a43fc2b1694823919e33c2dafa5037) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:02.573082 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 5af63f43d2f64606ab1becacf75367c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5af63f43d2f64606ab1becacf75367c9 already in progress: creating tablet
I20251028 03:00:02.574729 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet 5af63f43d2f64606ab1becacf75367c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 94a43fc2b1694823919e33c2dafa5037) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:02.575196 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 5af63f43d2f64606ab1becacf75367c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5af63f43d2f64606ab1becacf75367c9 already in progress: creating tablet
I20251028 03:00:02.575495 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 86cd4029953949f7994766653bf40b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:02.575934 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 86cd4029953949f7994766653bf40b7c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 86cd4029953949f7994766653bf40b7c already in progress: creating tablet
I20251028 03:00:02.576647 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_eb695c5a00aa482d958643fb13e4abaf" } new_table_name: "default.Table_49ace696414843d480acafa81bb4be89" modify_external_catalogs: true
I20251028 03:00:02.578255 8546 tablet_bootstrap.cc:492] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.578382 8546 ts_tablet_manager.cc:1403] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.001s
I20251028 03:00:02.578929 8546 raft_consensus.cc:359] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.579038 8546 raft_consensus.cc:385] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.579074 8546 raft_consensus.cc:740] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.579205 8546 consensus_queue.cc:260] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.579612 8546 ts_tablet_manager.cc:1434] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.579811 8546 tablet_bootstrap.cc:492] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.579833 10246 raft_consensus.cc:493] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.579921 10246 raft_consensus.cc:515] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.580313 10246 leader_election.cc:290] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.580816 8546 tablet_bootstrap.cc:654] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.581266 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e45fe8888fbc415ea637b363dccdd06b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.581393 8142 raft_consensus.cc:2468] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.582042 8354 leader_election.cc:304] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:02.582219 10245 raft_consensus.cc:2804] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.582262 10245 raft_consensus.cc:493] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.582302 10245 raft_consensus.cc:3060] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.582394 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e45fe8888fbc415ea637b363dccdd06b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:02.582762 8356 leader_election.cc:343] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.586309 10245 raft_consensus.cc:515] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.586822 10245 leader_election.cc:290] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.587095 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e45fe8888fbc415ea637b363dccdd06b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.587124 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e45fe8888fbc415ea637b363dccdd06b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:02.587231 8142 raft_consensus.cc:3060] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:02.587502 8356 leader_election.cc:343] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.588174 8545 tablet_bootstrap.cc:492] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.588256 8545 ts_tablet_manager.cc:1403] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.000s
I20251028 03:00:02.588478 8142 raft_consensus.cc:2468] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.588723 8545 raft_consensus.cc:359] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.588841 8545 raft_consensus.cc:385] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.588869 8545 raft_consensus.cc:740] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.588889 8354 leader_election.cc:304] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.588975 8545 consensus_queue.cc:260] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.589234 8545 ts_tablet_manager.cc:1434] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.589406 8545 tablet_bootstrap.cc:492] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.590382 8545 tablet_bootstrap.cc:654] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.593163 10245 raft_consensus.cc:2804] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.593283 10245 raft_consensus.cc:697] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:02.593401 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f454943605324eec83a70310f675f27f" } new_table_name: "default.Table_0d5bef57eadd45538258da2301f0cd31" modify_external_catalogs: true
I20251028 03:00:02.593430 10245 consensus_queue.cc:237] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.594190 8546 tablet_bootstrap.cc:492] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.594280 8546 ts_tablet_manager.cc:1403] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20251028 03:00:02.594271 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet f03b90d615ab435f9a244504d8d01660
I20251028 03:00:02.594763 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1a104ff3307b4f44bc0ee3bc31e8bfe7" } modify_external_catalogs: true
I20251028 03:00:02.594858 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1a104ff3307b4f44bc0ee3bc31e8bfe7" } modify_external_catalogs: true
I20251028 03:00:02.594905 8546 raft_consensus.cc:359] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.595022 8546 raft_consensus.cc:385] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.595055 8546 raft_consensus.cc:740] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.595444 8546 consensus_queue.cc:260] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.595738 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet f03b90d615ab435f9a244504d8d01660 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.596179 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d2e449df11e04f8fbf35620bdc11b19d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:02.596267 8546 ts_tablet_manager.cc:1434] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:02.596243 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet f03b90d615ab435f9a244504d8d01660 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:02.596557 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2e449df11e04f8fbf35620bdc11b19d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.597159 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet f03b90d615ab435f9a244504d8d01660 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:02.597599 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet f03b90d615ab435f9a244504d8d01660 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f03b90d615ab435f9a244504d8d01660 already in progress: creating tablet
W20251028 03:00:02.597918 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet f03b90d615ab435f9a244504d8d01660 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f03b90d615ab435f9a244504d8d01660 already in progress: creating tablet
I20251028 03:00:02.598032 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 7e0e038d33bf45f693da251cd560ae2a
I20251028 03:00:02.598273 7967 catalog_manager.cc:6719] ReplaceTablet: tablet 7e0e038d33bf45f693da251cd560ae2a deleted and replaced by tablet 5e649541935b458f8e1583c2db9e23f5
I20251028 03:00:02.598448 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_de56c060321c452aa62bcc0844c2f3d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.598810 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de56c060321c452aa62bcc0844c2f3d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.599875 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e0e038d33bf45f693da251cd560ae2a with delete_type TABLET_DATA_DELETED (replaced by tablet 5e649541935b458f8e1583c2db9e23f5) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.600108 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e0e038d33bf45f693da251cd560ae2a with delete_type TABLET_DATA_DELETED (replaced by tablet 5e649541935b458f8e1583c2db9e23f5) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.600545 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e0e038d33bf45f693da251cd560ae2a with delete_type TABLET_DATA_DELETED (replaced by tablet 5e649541935b458f8e1583c2db9e23f5) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:02.601202 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 7e0e038d33bf45f693da251cd560ae2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e0e038d33bf45f693da251cd560ae2a already in progress: creating tablet
W20251028 03:00:02.603495 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 7e0e038d33bf45f693da251cd560ae2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e0e038d33bf45f693da251cd560ae2a already in progress: creating tablet
I20251028 03:00:02.606426 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3aeb143d1cf14c2b8217326c4689dd59" } new_table_name: "default.Table_df2597e0d21d468d885a1805b730fe70" modify_external_catalogs: true
I20251028 03:00:02.613659 8546 tablet_bootstrap.cc:492] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.614761 8546 tablet_bootstrap.cc:654] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:02.615727 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet f03b90d615ab435f9a244504d8d01660 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f03b90d615ab435f9a244504d8d01660 already in progress: creating tablet
W20251028 03:00:02.615926 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 7e0e038d33bf45f693da251cd560ae2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e0e038d33bf45f693da251cd560ae2a already in progress: creating tablet
I20251028 03:00:02.616362 8546 tablet_bootstrap.cc:492] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.616456 8546 ts_tablet_manager.cc:1403] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20251028 03:00:02.616852 8546 raft_consensus.cc:359] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.616955 8546 raft_consensus.cc:385] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.616994 8546 raft_consensus.cc:740] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.617125 8546 consensus_queue.cc:260] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.617384 8546 ts_tablet_manager.cc:1434] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.617517 8546 tablet_bootstrap.cc:492] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.618259 8546 tablet_bootstrap.cc:654] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.619141 8546 tablet_bootstrap.cc:492] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.619227 8546 ts_tablet_manager.cc:1403] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:02.619656 8546 raft_consensus.cc:359] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.619766 8546 raft_consensus.cc:385] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.619829 8546 raft_consensus.cc:740] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.619984 8546 consensus_queue.cc:260] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.620321 8546 ts_tablet_manager.cc:1434] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.620898 8546 tablet_bootstrap.cc:492] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.621905 8546 tablet_bootstrap.cc:654] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.625656 10192 tablet.cc:1722] T e95a45cb612447dd9bebd9f7a019d1f9 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:02.627914 9909 tablet.cc:1722] T e95a45cb612447dd9bebd9f7a019d1f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:02.628641 8544 tablet_bootstrap.cc:492] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:02.628723 8544 ts_tablet_manager.cc:1403] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.175s user 0.000s sys 0.002s
I20251028 03:00:02.629189 8544 raft_consensus.cc:359] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.629304 8544 raft_consensus.cc:385] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.629336 8544 raft_consensus.cc:740] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.629451 8544 consensus_queue.cc:260] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.629745 8544 ts_tablet_manager.cc:1434] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:02.629935 8544 tablet_bootstrap.cc:492] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:02.630980 8544 tablet_bootstrap.cc:654] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.633996 8546 tablet_bootstrap.cc:492] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.634086 8546 ts_tablet_manager.cc:1403] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20251028 03:00:02.634610 8546 raft_consensus.cc:359] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.634721 8546 raft_consensus.cc:385] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.634753 8546 raft_consensus.cc:740] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.634900 8546 consensus_queue.cc:260] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.637652 8546 ts_tablet_manager.cc:1434] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 03:00:02.637903 8546 tablet_bootstrap.cc:492] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.639103 8546 tablet_bootstrap.cc:654] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.640630 10187 tablet.cc:1722] T e95a45cb612447dd9bebd9f7a019d1f9 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:02.633219 10286 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9343ca8510de4eaa9945786e054a866d. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 03:00:02.641429 10286 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 03:00:02.642210 8545 tablet_bootstrap.cc:492] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.642306 8545 ts_tablet_manager.cc:1403] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.053s user 0.000s sys 0.003s
I20251028 03:00:02.642791 8546 tablet_bootstrap.cc:492] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.642864 8546 ts_tablet_manager.cc:1403] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251028 03:00:02.642868 8545 raft_consensus.cc:359] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.642982 8545 raft_consensus.cc:385] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.643016 8545 raft_consensus.cc:740] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.643136 8545 consensus_queue.cc:260] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.643321 8546 raft_consensus.cc:359] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.643410 8546 raft_consensus.cc:385] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.643440 8546 raft_consensus.cc:740] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.643451 8545 ts_tablet_manager.cc:1434] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:02.643568 8546 consensus_queue.cc:260] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.643651 8545 tablet_bootstrap.cc:492] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.644097 8546 ts_tablet_manager.cc:1434] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.644256 8546 tablet_bootstrap.cc:492] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.630606 10056 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ccbdf478f18948d19de96574a685bfda" instance_seqno: 1761620389479519) as {username='slave'} at 127.7.154.130:39837; Asking this server to re-register.
I20251028 03:00:02.644708 8545 tablet_bootstrap.cc:654] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.633080 10057 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" instance_seqno: 1761620389689963) as {username='slave'} at 127.7.154.131:48127; Asking this server to re-register.
I20251028 03:00:02.645387 8546 tablet_bootstrap.cc:654] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.645638 8468 heartbeater.cc:461] Registering TS with master...
I20251028 03:00:02.639884 10282 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 03:00:02.647024 10282 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [sys.catalog]: This master's current role is: FOLLOWER
I20251028 03:00:02.648303 10057 ts_manager.cc:194] Registered new tserver with Master: d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:02.648367 8332 heartbeater.cc:461] Registering TS with master...
I20251028 03:00:02.648711 10056 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" instance_seqno: 1761620389287366) as {username='slave'} at 127.7.154.129:58529; Asking this server to re-register.
I20251028 03:00:02.649945 9884 consensus_queue.cc:1048] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.651212 8195 heartbeater.cc:461] Registering TS with master...
I20251028 03:00:02.652624 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet efbbfee9d8c44e5489e7a24fdd46f290
I20251028 03:00:02.653153 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_b487827beda64cfe9fc5cdf1daa37a4d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:02.653245 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet efbbfee9d8c44e5489e7a24fdd46f290 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.653276 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet efbbfee9d8c44e5489e7a24fdd46f290 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:02.653600 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b487827beda64cfe9fc5cdf1daa37a4d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:02.654006 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet efbbfee9d8c44e5489e7a24fdd46f290 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet efbbfee9d8c44e5489e7a24fdd46f290 already in progress: creating tablet
I20251028 03:00:02.654081 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet efbbfee9d8c44e5489e7a24fdd46f290 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:02.654479 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet efbbfee9d8c44e5489e7a24fdd46f290 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet efbbfee9d8c44e5489e7a24fdd46f290 already in progress: creating tablet
I20251028 03:00:02.654572 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e7689390da9341cc8efa3e3a38c15a47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.654922 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7689390da9341cc8efa3e3a38c15a47 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.655421 10057 ts_manager.cc:194] Registered new tserver with Master: fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:02.655794 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 848dbb23d5604a7085947771c5ee3d5e from {username='slave'} at 127.0.0.1:47364
W20251028 03:00:02.656229 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 110d3531524a460a8e6c2966ef0c367f consensus state has no leader
I20251028 03:00:02.661823 9958 raft_consensus.cc:493] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.661947 9958 raft_consensus.cc:515] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.662357 9958 leader_election.cc:290] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:02.664021 8220 leader_election.cc:343] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 9277a4ffd0ca4bf7afd29e1c6ab33bc9
I20251028 03:00:02.664309 8546 tablet_bootstrap.cc:492] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.664379 9884 raft_consensus.cc:493] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.664476 9884 raft_consensus.cc:515] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.664829 9884 leader_election.cc:290] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:02.664973 8546 ts_tablet_manager.cc:1403] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20251028 03:00:02.665426 8546 raft_consensus.cc:359] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.665488 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6599ec60c3004e349e7e576e4d13c4fe" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:02.665534 8546 raft_consensus.cc:385] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.665563 8546 raft_consensus.cc:740] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.665593 8420 raft_consensus.cc:2468] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:02.665738 8546 consensus_queue.cc:260] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.666015 8546 ts_tablet_manager.cc:1434] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.666216 8220 leader_election.cc:304] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.666345 9595 consensus_queue.cc:1048] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.666546 9595 raft_consensus.cc:2804] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.666582 9595 raft_consensus.cc:493] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.666613 9595 raft_consensus.cc:3060] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.667641 9595 raft_consensus.cc:515] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.667974 9595 leader_election.cc:290] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:02.669912 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6599ec60c3004e349e7e576e4d13c4fe" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:02.670032 8416 raft_consensus.cc:3060] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.671012 8416 raft_consensus.cc:2468] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:02.672113 8220 leader_election.cc:304] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.672277 8553 raft_consensus.cc:2804] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.672343 8553 raft_consensus.cc:697] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:02.672465 8553 consensus_queue.cc:237] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 03:00:02.673820 8218 leader_election.cc:343] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 9277a4ffd0ca4bf7afd29e1c6ab33bc9
I20251028 03:00:02.673899 8218 leader_election.cc:304] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 03:00:02.674153 8218 leader_election.cc:343] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 6599ec60c3004e349e7e576e4d13c4fe
I20251028 03:00:02.674404 9595 raft_consensus.cc:2749] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251028 03:00:02.676870 8218 leader_election.cc:343] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 6599ec60c3004e349e7e576e4d13c4fe
I20251028 03:00:02.678725 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_49ec85dd85624ee5bd983a6beddace4e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.680480 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49ec85dd85624ee5bd983a6beddace4e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.684985 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0f7ea0b3b42f4d61b7970b0dfcd52e4d" } modify_external_catalogs: true
I20251028 03:00:02.685127 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0f7ea0b3b42f4d61b7970b0dfcd52e4d" } modify_external_catalogs: true
I20251028 03:00:02.688041 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_01472823184b4a239a78ce504161be41" } new_table_name: "default.Table_2e67a0ecccde42c8bff633eeba8af217" modify_external_catalogs: true
I20251028 03:00:02.694204 8546 tablet_bootstrap.cc:492] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.695083 10246 raft_consensus.cc:493] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.695185 10246 raft_consensus.cc:515] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.695571 8546 tablet_bootstrap.cc:654] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.695660 10246 leader_election.cc:290] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.699609 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b7d3062426434c339be4a745decae729" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:02.700014 8356 leader_election.cc:343] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:02.700655 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet efbbfee9d8c44e5489e7a24fdd46f290 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet efbbfee9d8c44e5489e7a24fdd46f290 already in progress: creating tablet
I20251028 03:00:02.703336 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_6df12dce729d47de9eb4760fec2119de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.703819 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6df12dce729d47de9eb4760fec2119de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.704516 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b7d3062426434c339be4a745decae729" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.704650 8146 raft_consensus.cc:2468] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:02.706115 10309 catalog_manager.cc:1269] Loaded cluster ID: a891460fa9a9449bb0459a49a032f9be
I20251028 03:00:02.706202 10309 catalog_manager.cc:1562] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: loading cluster ID for follower catalog manager: success
W20251028 03:00:02.706610 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7ee583e318174f34b162c2a104a0a3a9 consensus state has no leader
I20251028 03:00:02.706921 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ad4030fee54c4132b2fba53c521bedd6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:02.710865 8354 leader_election.cc:304] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.711205 10246 raft_consensus.cc:2804] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.711278 10246 raft_consensus.cc:493] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.711342 10246 raft_consensus.cc:3060] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:02.710517 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad4030fee54c4132b2fba53c521bedd6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.712574 10246 raft_consensus.cc:515] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.712970 10246 leader_election.cc:290] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.713912 8285 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b7d3062426434c339be4a745decae729" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:02.713970 8553 raft_consensus.cc:493] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.714057 8553 raft_consensus.cc:515] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.714385 8553 leader_election.cc:290] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:02.714448 8356 leader_election.cc:343] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.714843 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12d103e9b5024a87aa232c26e0c51ab9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:02.714962 8420 raft_consensus.cc:2468] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:02.715407 8220 leader_election.cc:304] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.715621 8553 raft_consensus.cc:2804] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.715672 8553 raft_consensus.cc:493] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.715706 8553 raft_consensus.cc:3060] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.716135 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b7d3062426434c339be4a745decae729" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.716246 8148 raft_consensus.cc:3060] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.717000 10309 catalog_manager.cc:1584] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: acquiring CA information for follower catalog manager: success
I20251028 03:00:02.717231 8148 raft_consensus.cc:2468] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:02.717809 10057 ts_manager.cc:194] Registered new tserver with Master: ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.718127 8553 raft_consensus.cc:515] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.718811 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12d103e9b5024a87aa232c26e0c51ab9" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:02.718917 8420 raft_consensus.cc:3060] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.719858 8420 raft_consensus.cc:2468] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:02.720243 8354 leader_election.cc:304] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.720244 8220 leader_election.cc:304] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.720412 9595 raft_consensus.cc:2804] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.720489 9595 raft_consensus.cc:697] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:02.720620 9595 consensus_queue.cc:237] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.720685 10246 raft_consensus.cc:2804] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.720764 10246 raft_consensus.cc:697] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:02.720880 10246 consensus_queue.cc:237] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:02.721866 8218 leader_election.cc:343] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 12d103e9b5024a87aa232c26e0c51ab9
I20251028 03:00:02.722059 8553 leader_election.cc:290] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:02.722357 8545 tablet_bootstrap.cc:492] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.722442 8545 ts_tablet_manager.cc:1403] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.079s user 0.002s sys 0.000s
I20251028 03:00:02.722914 8545 raft_consensus.cc:359] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.723006 8545 raft_consensus.cc:385] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.723038 8545 raft_consensus.cc:740] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.723121 8545 consensus_queue.cc:260] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.723412 8545 ts_tablet_manager.cc:1434] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.723819 8545 tablet_bootstrap.cc:492] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.724021 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ad274b5a83e045a685e2d4322bde36fb" } modify_external_catalogs: true
I20251028 03:00:02.724139 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ad274b5a83e045a685e2d4322bde36fb" } modify_external_catalogs: true
I20251028 03:00:02.725826 8545 tablet_bootstrap.cc:654] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.726769 8546 tablet_bootstrap.cc:492] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:02.726861 8546 ts_tablet_manager.cc:1403] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.000s
I20251028 03:00:02.727389 8546 raft_consensus.cc:359] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.727483 8546 raft_consensus.cc:385] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.727511 8546 raft_consensus.cc:740] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.727623 8546 consensus_queue.cc:260] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.727909 8546 ts_tablet_manager.cc:1434] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:02.728070 8546 tablet_bootstrap.cc:492] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:02.730041 9595 raft_consensus.cc:493] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.730127 9595 raft_consensus.cc:515] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.730464 9595 leader_election.cc:290] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:02.731153 8220 leader_election.cc:343] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 6bc54972260747f3af0c18d05c6a3442
I20251028 03:00:02.731536 8546 tablet_bootstrap.cc:654] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:02.734747 8218 leader_election.cc:343] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 12d103e9b5024a87aa232c26e0c51ab9
W20251028 03:00:02.735149 8218 leader_election.cc:343] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 6bc54972260747f3af0c18d05c6a3442
I20251028 03:00:02.735220 8218 leader_election.cc:304] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:02.735538 8553 raft_consensus.cc:2749] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.735890 8545 tablet_bootstrap.cc:492] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.736043 8545 ts_tablet_manager.cc:1403] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 03:00:02.736543 8545 raft_consensus.cc:359] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:02.736580 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5584a2f654f14984b2d85d2c44dbc692"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:02.736711 8545 raft_consensus.cc:385] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.736752 8545 raft_consensus.cc:740] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.736912 8545 consensus_queue.cc:260] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 03:00:02.737494 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5584a2f654f14984b2d85d2c44dbc692 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.739799 8544 tablet_bootstrap.cc:492] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:02.739897 8544 ts_tablet_manager.cc:1403] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.110s user 0.000s sys 0.002s
I20251028 03:00:02.740859 8544 raft_consensus.cc:359] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.740981 8544 raft_consensus.cc:385] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.741014 8544 raft_consensus.cc:740] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.741128 8544 consensus_queue.cc:260] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.741416 8544 ts_tablet_manager.cc:1434] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:02.741775 8544 tablet_bootstrap.cc:492] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:02.742017 8545 ts_tablet_manager.cc:1434] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251028 03:00:02.742223 8545 tablet_bootstrap.cc:492] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.743175 8544 tablet_bootstrap.cc:654] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.743399 8545 tablet_bootstrap.cc:654] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.744975 9190 consensus_queue.cc:1048] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.753038 10309 catalog_manager.cc:1612] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251028 03:00:02.753990 8545 tablet_bootstrap.cc:492] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:02.754194 8545 ts_tablet_manager.cc:1403] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20251028 03:00:02.754813 8545 raft_consensus.cc:359] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.755007 8545 raft_consensus.cc:385] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:02.755084 8545 raft_consensus.cc:740] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:02.755350 8545 consensus_queue.cc:260] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.756891 7962 catalog_manager.cc:5649] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.757118 7962 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet f422030a6a4042009efc7fcfb21a6bca (table default.Table_a01151cafd944efcbc12e5f654a7a05a [id=fdd6ecd598f240948882d710fd485da2]). Expected version 2 got 0
I20251028 03:00:02.757340 7962 catalog_manager.cc:5649] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.757541 7962 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 9a81841c22ae49c0a17931d90407accc (table default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]). Expected version 1 got 0
I20251028 03:00:02.757649 7962 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 5c1994484abf4b6fbbc3f47527558cdb (table default.Table_f454943605324eec83a70310f675f27f [id=b0407c8f78a84245a954c67e888da3ec]). Expected version 4 got 0
I20251028 03:00:02.757771 7962 catalog_manager.cc:5649] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.757956 7962 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 338cf9d02142436f97ccd44313cc8199 (table default.Table_77a37bbdfa4f41458e6b50fb213850bc [id=5f61b3c12ed647968fc2932aa24a13f3]). Expected version 2 got 0
W20251028 03:00:02.758435 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5c1994484abf4b6fbbc3f47527558cdb consensus state has no leader
I20251028 03:00:02.758502 8553 raft_consensus.cc:493] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.758608 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet a7fa9f23989d4bdba6545d0e681c78e5 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:02.758600 8553 raft_consensus.cc:515] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.758690 8545 ts_tablet_manager.cc:1434] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 03:00:02.758880 8545 tablet_bootstrap.cc:492] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:02.759004 8553 leader_election.cc:290] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:02.759442 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c0081e0ce4d4e499c37f217fe62c546" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:02.759562 8420 raft_consensus.cc:2468] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:02.759838 8220 leader_election.cc:304] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.760176 8553 raft_consensus.cc:2804] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.760236 8553 raft_consensus.cc:493] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.760277 8553 raft_consensus.cc:3060] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.760711 8545 tablet_bootstrap.cc:654] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:02.761471 8553 raft_consensus.cc:515] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.761840 8553 leader_election.cc:290] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:02.762095 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_28717f0c640748cca2efe4a69c6b3a2b" } modify_external_catalogs: true
I20251028 03:00:02.762202 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_28717f0c640748cca2efe4a69c6b3a2b" } modify_external_catalogs: true
I20251028 03:00:02.762270 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c0081e0ce4d4e499c37f217fe62c546" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:02.762375 8420 raft_consensus.cc:3060] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.763837 8420 raft_consensus.cc:2468] T 4c0081e0ce4d4e499c37f217fe62c546 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:02.764125 8220 leader_election.cc:304] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:02.764603 8553 raft_consensus.cc:2804] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.764777 8553 raft_consensus.cc:697] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:02.764916 8553 consensus_queue.cc:237] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.765434 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c0081e0ce4d4e499c37f217fe62c546" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.765583 8148 raft_consensus.cc:2468] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:02.765872 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 87c0f1accc254ee19372bcff9a71ed5d
I20251028 03:00:02.766461 8122 tablet_service.cc:1505] Processing CreateTablet for tablet bbdd209e156c482a8910379e73283f4b (DEFAULT_TABLE table=default.Table_2687d010e9e444698c450ff0db40f8da [id=95b5925b192243f793c2f1209be8e491]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.766511 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 87c0f1accc254ee19372bcff9a71ed5d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.766633 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 87c0f1accc254ee19372bcff9a71ed5d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.766769 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbdd209e156c482a8910379e73283f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.767038 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet 87c0f1accc254ee19372bcff9a71ed5d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:02.767361 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 87c0f1accc254ee19372bcff9a71ed5d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87c0f1accc254ee19372bcff9a71ed5d already in progress: creating tablet
W20251028 03:00:02.767616 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 87c0f1accc254ee19372bcff9a71ed5d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87c0f1accc254ee19372bcff9a71ed5d already in progress: creating tablet
I20251028 03:00:02.768196 8263 tablet_service.cc:1505] Processing CreateTablet for tablet bbdd209e156c482a8910379e73283f4b (DEFAULT_TABLE table=default.Table_2687d010e9e444698c450ff0db40f8da [id=95b5925b192243f793c2f1209be8e491]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.768455 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbdd209e156c482a8910379e73283f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.769500 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 848dbb23d5604a7085947771c5ee3d5e
I20251028 03:00:02.769794 7967 catalog_manager.cc:6719] ReplaceTablet: tablet 848dbb23d5604a7085947771c5ee3d5e deleted and replaced by tablet 83639e6b4b9e454ca3ce56903d2d0337
I20251028 03:00:02.770682 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 04723e51ee66454182da19634c14ffb4
I20251028 03:00:02.770772 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4c0081e0ce4d4e499c37f217fe62c546" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:02.770900 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 04723e51ee66454182da19634c14ffb4 deleted and replaced by tablet 67c0d8776ff94da1a2249d5a288568da
I20251028 03:00:02.770887 8142 raft_consensus.cc:3060] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.771531 8392 tablet_service.cc:1505] Processing CreateTablet for tablet bbdd209e156c482a8910379e73283f4b (DEFAULT_TABLE table=default.Table_2687d010e9e444698c450ff0db40f8da [id=95b5925b192243f793c2f1209be8e491]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.771801 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbdd209e156c482a8910379e73283f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.772038 8142 raft_consensus.cc:2468] T 4c0081e0ce4d4e499c37f217fe62c546 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:02.772593 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 2a57c473fcd74b92819105d83f52b3a2
I20251028 03:00:02.775333 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet f422030a6a4042009efc7fcfb21a6bca (table default.Table_a01151cafd944efcbc12e5f654a7a05a [id=fdd6ecd598f240948882d710fd485da2]). Expected version 2 got 0
I20251028 03:00:02.775470 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 8665d2e933fa423d81a1b32f2ce8629b (table default.Table_139255bc4fb74fae8fb86563b54826bd [id=dc3721126b09466ab5efdb8bd4b46d89]). Expected version 1 got 0
I20251028 03:00:02.775548 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 9a81841c22ae49c0a17931d90407accc (table default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]). Expected version 1 got 0
I20251028 03:00:02.775591 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet 04723e51ee66454182da19634c14ffb4 with delete_type TABLET_DATA_DELETED (replaced by tablet 67c0d8776ff94da1a2249d5a288568da) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.775679 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 2a57c473fcd74b92819105d83f52b3a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:02.776386 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 87c0f1accc254ee19372bcff9a71ed5d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87c0f1accc254ee19372bcff9a71ed5d already in progress: creating tablet
I20251028 03:00:02.776324 7969 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_d51065c1d57c4857bc5765d1baf9da06 [id=3815e9736bd34d668ec7f7e113e11e6f] alter complete (version 1)
I20251028 03:00:02.776950 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 04723e51ee66454182da19634c14ffb4 with delete_type TABLET_DATA_DELETED (replaced by tablet 67c0d8776ff94da1a2249d5a288568da) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.776948 10075 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40286 (request call id 0) took 1867 ms. Trace:
I20251028 03:00:02.777098 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet 2a57c473fcd74b92819105d83f52b3a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.777575 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 848dbb23d5604a7085947771c5ee3d5e with delete_type TABLET_DATA_DELETED (replaced by tablet 83639e6b4b9e454ca3ce56903d2d0337) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.778973 9788 raft_consensus.cc:493] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.779069 9788 raft_consensus.cc:515] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.779484 9788 leader_election.cc:290] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.779646 8263 tablet_service.cc:1552] Processing DeleteTablet for tablet 848dbb23d5604a7085947771c5ee3d5e with delete_type TABLET_DATA_DELETED (replaced by tablet 83639e6b4b9e454ca3ce56903d2d0337) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.779687 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 04723e51ee66454182da19634c14ffb4 with delete_type TABLET_DATA_DELETED (replaced by tablet 67c0d8776ff94da1a2249d5a288568da) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.779783 8260 tablet_service.cc:1552] Processing DeleteTablet for tablet 2a57c473fcd74b92819105d83f52b3a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.780328 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1c41d49f9eba40a48bd6900671297db9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:02.780453 8420 raft_consensus.cc:2468] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:02.780663 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1c41d49f9eba40a48bd6900671297db9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.780786 8083 leader_election.cc:304] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:02.781051 9788 raft_consensus.cc:2804] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.781116 9788 raft_consensus.cc:493] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.781164 9788 raft_consensus.cc:3060] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.782270 9788 raft_consensus.cc:515] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.782632 9788 leader_election.cc:290] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:02.782984 8083 leader_election.cc:343] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.783495 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1c41d49f9eba40a48bd6900671297db9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:02.784021 8083 leader_election.cc:343] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:02.784552 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 04723e51ee66454182da19634c14ffb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04723e51ee66454182da19634c14ffb4 already in progress: creating tablet
W20251028 03:00:02.784701 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 2a57c473fcd74b92819105d83f52b3a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a57c473fcd74b92819105d83f52b3a2 already in progress: creating tablet
I20251028 03:00:02.784672 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1c41d49f9eba40a48bd6900671297db9" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:02.784793 8420 raft_consensus.cc:3060] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:02.784814 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 848dbb23d5604a7085947771c5ee3d5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 848dbb23d5604a7085947771c5ee3d5e already in progress: creating tablet
I20251028 03:00:02.785085 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet fd8c04efd8c2456b9c7643641c7fbc0d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.785300 8550 tablet_replica.cc:333] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:02.785449 8550 raft_consensus.cc:2243] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.785547 8550 raft_consensus.cc:2272] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.785887 8420 raft_consensus.cc:2468] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:02.786093 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 36cfa812c550475385980578108fb6be
I20251028 03:00:02.786482 9190 raft_consensus.cc:493] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.786535 8550 ts_tablet_manager.cc:1916] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.786576 9190 raft_consensus.cc:515] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.786943 9190 leader_election.cc:290] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.788755 8550 ts_tablet_manager.cc:1929] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.788810 8550 log.cc:1199] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/fd8c04efd8c2456b9c7643641c7fbc0d
I20251028 03:00:02.788968 8550 ts_tablet_manager.cc:1950] T fd8c04efd8c2456b9c7643641c7fbc0d P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:02.790113 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f39ed19223fa4268a01aaf4c7762d86e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:02.790588 8354 leader_election.cc:343] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.791550 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f39ed19223fa4268a01aaf4c7762d86e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.791620 8083 leader_election.cc:304] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
W20251028 03:00:02.791971 8356 leader_election.cc:343] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.792047 8356 leader_election.cc:304] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:02.792240 9190 raft_consensus.cc:2749] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.795619 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 848dbb23d5604a7085947771c5ee3d5e with delete_type TABLET_DATA_DELETED (replaced by tablet 83639e6b4b9e454ca3ce56903d2d0337) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.795997 9788 raft_consensus.cc:2804] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.796485 9788 raft_consensus.cc:697] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:02.796617 9788 consensus_queue.cc:237] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.799860 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet fd8c04efd8c2456b9c7643641c7fbc0d (table default.Table_327424a7e56243ae97d58c0f6fd560fd [id=009632a3749c4bbe9cf4e856758b9a72]) successfully deleted
I20251028 03:00:02.802044 7964 catalog_manager.cc:5649] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.802279 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet ad926e9ee0044c62a7abbfca959a4f56 (table default.Table_6ad31a79d9f043628bdac1dd2d4c263c [id=4570d7ddf33c447fb40d6411ea47e52a]). Expected version 1 got 0
I20251028 03:00:02.802387 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet f422030a6a4042009efc7fcfb21a6bca (table default.Table_a01151cafd944efcbc12e5f654a7a05a [id=fdd6ecd598f240948882d710fd485da2]). Expected version 2 got 0
I20251028 03:00:02.802460 7964 catalog_manager.cc:5649] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.802592 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 5c1994484abf4b6fbbc3f47527558cdb (table default.Table_0d5bef57eadd45538258da2301f0cd31 [id=b0407c8f78a84245a954c67e888da3ec]). Expected version 5 got 0
I20251028 03:00:02.802680 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet e3b33c4d189247a89878b6a8e1fcbac0 (table default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]). Expected version 1 got 0
I20251028 03:00:02.802757 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 2bf100407b034626b4a9f7e2340f9cf0 (table default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262]). Expected version 2 got 0
I20251028 03:00:02.803848 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 36cfa812c550475385980578108fb6be with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.803947 9190 consensus_queue.cc:1048] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.806780 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5b46e89d6c094f008e7da4d4ae8b6c7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:02.806887 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 36cfa812c550475385980578108fb6be with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.806973 9190 consensus_queue.cc:1048] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 03:00:02.807212 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b46e89d6c094f008e7da4d4ae8b6c7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.808439 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7cc757dea1fe4568957409286675204d" } modify_external_catalogs: true
I20251028 03:00:02.808543 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7cc757dea1fe4568957409286675204d" } modify_external_catalogs: true
I20251028 03:00:02.808995 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5fc1ce4002024900b764d32451fe2695"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:02.809335 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5fc1ce4002024900b764d32451fe2695 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:02.810405 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 88cb16241e364f56b251d88079936c7b from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:02.811251 9595 raft_consensus.cc:493] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.811371 9595 raft_consensus.cc:515] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.811728 9595 leader_election.cc:290] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:02.812286 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "338cf9d02142436f97ccd44313cc8199" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:02.812448 8148 raft_consensus.cc:2393] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 03:00:02.812969 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "338cf9d02142436f97ccd44313cc8199" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:02.812987 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b12ebf9b73404d62b93a0eefa8d10b53" } new_table_name: "default.Table_ee2008b68e1e4ae5a0f7034250a07c0c" modify_external_catalogs: true
W20251028 03:00:02.813798 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 848dbb23d5604a7085947771c5ee3d5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 848dbb23d5604a7085947771c5ee3d5e already in progress: creating tablet
W20251028 03:00:02.813961 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 2a57c473fcd74b92819105d83f52b3a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a57c473fcd74b92819105d83f52b3a2 already in progress: creating tablet
I20251028 03:00:02.814409 8220 leader_election.cc:304] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:02.814618 9595 raft_consensus.cc:3060] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.815845 9595 raft_consensus.cc:2749] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251028 03:00:02.816335 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 04723e51ee66454182da19634c14ffb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04723e51ee66454182da19634c14ffb4 already in progress: creating tablet
W20251028 03:00:02.816507 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 36cfa812c550475385980578108fb6be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36cfa812c550475385980578108fb6be already in progress: creating tablet
W20251028 03:00:02.816663 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 04723e51ee66454182da19634c14ffb4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04723e51ee66454182da19634c14ffb4 already in progress: creating tablet
W20251028 03:00:02.816812 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 2a57c473fcd74b92819105d83f52b3a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a57c473fcd74b92819105d83f52b3a2 already in progress: creating tablet
W20251028 03:00:02.816939 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 848dbb23d5604a7085947771c5ee3d5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 848dbb23d5604a7085947771c5ee3d5e already in progress: creating tablet
I20251028 03:00:02.817327 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 36cfa812c550475385980578108fb6be with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:02.817824 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 36cfa812c550475385980578108fb6be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36cfa812c550475385980578108fb6be already in progress: creating tablet
I20251028 03:00:02.818852 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_b2cfdddecdf14efcb61d58a4646b792f [id=25d0c2f2faef473b94ce2621281c08b0] alter complete (version 2)
I20251028 03:00:02.827479 10242 consensus_queue.cc:1048] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.777038 10075 rpcz_store.cc:276] 1028 03:00:00.909943 (+ 0us) service_pool.cc:168] Inserting onto call queue
1028 03:00:00.909990 (+ 47us) service_pool.cc:225] Handling call
1028 03:00:02.571984 (+1661994us) raft_consensus.cc:1471] Updating replica for 183 ops
1028 03:00:02.573582 (+ 1598us) raft_consensus.cc:1534] Early marking committed up to index 0
1028 03:00:02.573588 (+ 6us) raft_consensus.cc:1539] Triggering prepare for 183 ops
1028 03:00:02.574091 (+ 503us) write_op.cc:270] Start()
1028 03:00:02.574136 (+ 45us) write_op.cc:276] Timestamp: P: 1761620398877480 usec, L: 0
1028 03:00:02.575153 (+ 1017us) write_op.cc:270] Start()
1028 03:00:02.575178 (+ 25us) write_op.cc:276] Timestamp: P: 1761620398877986 usec, L: 0
1028 03:00:02.575349 (+ 171us) write_op.cc:270] Start()
1028 03:00:02.575374 (+ 25us) write_op.cc:276] Timestamp: P: 1761620398878369 usec, L: 0
1028 03:00:02.575471 (+ 97us) write_op.cc:270] Start()
1028 03:00:02.575484 (+ 13us) write_op.cc:276] Timestamp: P: 1761620398878896 usec, L: 0
1028 03:00:02.575577 (+ 93us) write_op.cc:270] Start()
1028 03:00:02.575594 (+ 17us) write_op.cc:276] Timestamp: P: 1761620398881232 usec, L: 0
1028 03:00:02.575690 (+ 96us) write_op.cc:270] Start()
1028 03:00:02.575703 (+ 13us) write_op.cc:276] Timestamp: P: 1761620398881571 usec, L: 0
1028 03:00:02.575796 (+ 93us) write_op.cc:270] Start()
1028 03:00:02.575827 (+ 31us) write_op.cc:276] Timestamp: P: 1761620398885506 usec, L: 0
1028 03:00:02.575923 (+ 96us) write_op.cc:270] Start()
1028 03:00:02.575936 (+ 13us) write_op.cc:276] Timestamp: P: 1761620398885974 usec, L: 0
1028 03:00:02.576031 (+ 95us) write_op.cc:270] Start()
1028 03:00:02.576043 (+ 12us) write_op.cc:276] Timestamp: P: 1761620398887997 usec, L: 0
1028 03:00:02.576118 (+ 75us) write_op.cc:270] Start()
1028 03:00:02.576128 (+ 10us) write_op.cc:276] Timestamp: P: 1761620398890037 usec, L: 0
1028 03:00:02.576199 (+ 71us) write_op.cc:270] Start()
1028 03:00:02.576209 (+ 10us) write_op.cc:276] Timestamp: P: 1761620398917981 usec, L: 0
1028 03:00:02.576273 (+ 64us) write_op.cc:270] Start()
1028 03:00:02.576285 (+ 12us) write_op.cc:276] Timestamp: P: 1761620398918441 usec, L: 0
1028 03:00:02.576359 (+ 74us) write_op.cc:270] Start()
1028 03:00:02.576371 (+ 12us) write_op.cc:276] Timestamp: P: 1761620398918818 usec, L: 0
1028 03:00:02.576449 (+ 78us) write_op.cc:270] Start()
1028 03:00:02.576459 (+ 10us) write_op.cc:276] Timestamp: P: 1761620398927418 usec, L: 0
1028 03:00:02.576529 (+ 70us) write_op.cc:270] Start()
1028 03:00:02.576540 (+ 11us) write_op.cc:276] Timestamp: P: 1761620398927842 usec, L: 0
1028 03:00:02.576618 (+ 78us) write_op.cc:270] Start()
1028 03:00:02.576629 (+ 11us) write_op.cc:276] Timestamp: P: 1761620398932017 usec, L: 0
1028 03:00:02.576706 (+ 77us) write_op.cc:270] Start()
1028 03:00:02.576722 (+ 16us) write_op.cc:276] Timestamp: P: 1761620398933914 usec, L: 0
1028 03:00:02.576829 (+ 107us) write_op.cc:270] Start()
1028 03:00:02.576845 (+ 16us) write_op.cc:276] Timestamp: P: 1761620398934290 usec, L: 0
1028 03:00:02.576935 (+ 90us) write_op.cc:270] Start()
1028 03:00:02.576947 (+ 12us) write_op.cc:276] Timestamp: P: 1761620398934674 usec, L: 0
1028 03:00:02.577038 (+ 91us) write_op.cc:270] Start()
1028 03:00:02.577051 (+ 13us) write_op.cc:276] Timestamp: P: 1761620398946879 usec, L: 0
1028 03:00:02.577130 (+ 79us) write_op.cc:270] Start()
1028 03:00:02.577142 (+ 12us) write_op.cc:276] Timestamp: P: 1761620398971516 usec, L: 0
1028 03:00:02.577230 (+ 88us) write_op.cc:270] Start()
1028 03:00:02.577244 (+ 14us) write_op.cc:276] Timestamp: P: 1761620399009318 usec, L: 0
1028 03:00:02.577322 (+ 78us) write_op.cc:270] Start()
1028 03:00:02.577339 (+ 17us) write_op.cc:276] Timestamp: P: 1761620399088964 usec, L: 0
1028 03:00:02.577423 (+ 84us) write_op.cc:270] Start()
1028 03:00:02.577436 (+ 13us) write_op.cc:276] Timestamp: P: 1761620399109518 usec, L: 0
1028 03:00:02.577536 (+ 100us) write_op.cc:270] Start()
1028 03:00:02.577550 (+ 14us) write_op.cc:276] Timestamp: P: 1761620399138514 usec, L: 0
1028 03:00:02.577640 (+ 90us) write_op.cc:270] Start()
1028 03:00:02.577653 (+ 13us) write_op.cc:276] Timestamp: P: 1761620399143921 usec, L: 0
1028 03:00:02.577734 (+ 81us) write_op.cc:270] Start()
1028 03:00:02.577749 (+ 15us) write_op.cc:276] Timestamp: P: 1761620399144454 usec, L: 0
1028 03:00:02.577838 (+ 89us) write_op.cc:270] Start()
1028 03:00:02.577849 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399144877 usec, L: 0
1028 03:00:02.577944 (+ 95us) write_op.cc:270] Start()
1028 03:00:02.577960 (+ 16us) write_op.cc:276] Timestamp: P: 1761620399158918 usec, L: 0
1028 03:00:02.578078 (+ 118us) write_op.cc:270] Start()
1028 03:00:02.578098 (+ 20us) write_op.cc:276] Timestamp: P: 1761620399159443 usec, L: 0
1028 03:00:02.578627 (+ 529us) write_op.cc:270] Start()
1028 03:00:02.578646 (+ 19us) write_op.cc:276] Timestamp: P: 1761620399174514 usec, L: 0
1028 03:00:02.578738 (+ 92us) write_op.cc:270] Start()
1028 03:00:02.578752 (+ 14us) write_op.cc:276] Timestamp: P: 1761620399174935 usec, L: 0
1028 03:00:02.578834 (+ 82us) write_op.cc:270] Start()
1028 03:00:02.578846 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399175364 usec, L: 0
1028 03:00:02.578933 (+ 87us) write_op.cc:270] Start()
1028 03:00:02.578951 (+ 18us) write_op.cc:276] Timestamp: P: 1761620399175701 usec, L: 0
1028 03:00:02.579063 (+ 112us) write_op.cc:270] Start()
1028 03:00:02.579078 (+ 15us) write_op.cc:276] Timestamp: P: 1761620399176188 usec, L: 0
1028 03:00:02.579162 (+ 84us) write_op.cc:270] Start()
1028 03:00:02.579176 (+ 14us) write_op.cc:276] Timestamp: P: 1761620399203956 usec, L: 0
1028 03:00:02.579257 (+ 81us) write_op.cc:270] Start()
1028 03:00:02.579276 (+ 19us) write_op.cc:276] Timestamp: P: 1761620399204491 usec, L: 0
1028 03:00:02.582775 (+ 3499us) write_op.cc:270] Start()
1028 03:00:02.582797 (+ 22us) write_op.cc:276] Timestamp: P: 1761620399205262 usec, L: 0
1028 03:00:02.582895 (+ 98us) write_op.cc:270] Start()
1028 03:00:02.582907 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399205864 usec, L: 0
1028 03:00:02.582990 (+ 83us) write_op.cc:270] Start()
1028 03:00:02.583015 (+ 25us) write_op.cc:276] Timestamp: P: 1761620399206440 usec, L: 0
1028 03:00:02.583098 (+ 83us) write_op.cc:270] Start()
1028 03:00:02.583116 (+ 18us) write_op.cc:276] Timestamp: P: 1761620399206963 usec, L: 0
1028 03:00:02.583259 (+ 143us) write_op.cc:270] Start()
1028 03:00:02.583281 (+ 22us) write_op.cc:276] Timestamp: P: 1761620399207917 usec, L: 0
1028 03:00:02.583402 (+ 121us) write_op.cc:270] Start()
1028 03:00:02.583415 (+ 13us) write_op.cc:276] Timestamp: P: 1761620399209603 usec, L: 0
1028 03:00:02.583483 (+ 68us) write_op.cc:270] Start()
1028 03:00:02.583493 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399210041 usec, L: 0
1028 03:00:02.583554 (+ 61us) write_op.cc:270] Start()
1028 03:00:02.583564 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399223585 usec, L: 0
1028 03:00:02.583638 (+ 74us) write_op.cc:270] Start()
1028 03:00:02.583648 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399224334 usec, L: 0
1028 03:00:02.583717 (+ 69us) write_op.cc:270] Start()
1028 03:00:02.583728 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399224801 usec, L: 0
1028 03:00:02.583791 (+ 63us) write_op.cc:270] Start()
1028 03:00:02.583803 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399239953 usec, L: 0
1028 03:00:02.583864 (+ 61us) write_op.cc:270] Start()
1028 03:00:02.583874 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399259996 usec, L: 0
1028 03:00:02.583948 (+ 74us) write_op.cc:270] Start()
1028 03:00:02.583958 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399307656 usec, L: 0
1028 03:00:02.584089 (+ 131us) write_op.cc:270] Start()
1028 03:00:02.584101 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399328982 usec, L: 0
1028 03:00:02.584167 (+ 66us) write_op.cc:270] Start()
1028 03:00:02.584177 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399340049 usec, L: 0
1028 03:00:02.584238 (+ 61us) write_op.cc:270] Start()
1028 03:00:02.584249 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399364410 usec, L: 0
1028 03:00:02.584320 (+ 71us) write_op.cc:270] Start()
1028 03:00:02.584330 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399364912 usec, L: 0
1028 03:00:02.584404 (+ 74us) write_op.cc:270] Start()
1028 03:00:02.584415 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399381406 usec, L: 0
1028 03:00:02.584489 (+ 74us) write_op.cc:270] Start()
1028 03:00:02.584499 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399381854 usec, L: 0
1028 03:00:02.584562 (+ 63us) write_op.cc:270] Start()
1028 03:00:02.584572 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399388267 usec, L: 0
1028 03:00:02.584644 (+ 72us) write_op.cc:270] Start()
1028 03:00:02.584654 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399393090 usec, L: 0
1028 03:00:02.584716 (+ 62us) write_op.cc:270] Start()
1028 03:00:02.584726 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399403523 usec, L: 0
1028 03:00:02.584791 (+ 65us) write_op.cc:270] Start()
1028 03:00:02.584821 (+ 30us) write_op.cc:276] Timestamp: P: 1761620399403980 usec, L: 0
1028 03:00:02.584890 (+ 69us) write_op.cc:270] Start()
1028 03:00:02.584901 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399404441 usec, L: 0
1028 03:00:02.584987 (+ 86us) write_op.cc:270] Start()
1028 03:00:02.585007 (+ 20us) write_op.cc:276] Timestamp: P: 1761620399404889 usec, L: 0
1028 03:00:02.585087 (+ 80us) write_op.cc:270] Start()
1028 03:00:02.585098 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399426263 usec, L: 0
1028 03:00:02.585158 (+ 60us) write_op.cc:270] Start()
1028 03:00:02.585168 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399429933 usec, L: 0
1028 03:00:02.585247 (+ 79us) write_op.cc:270] Start()
1028 03:00:02.585257 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399437060 usec, L: 0
1028 03:00:02.585332 (+ 75us) write_op.cc:270] Start()
1028 03:00:02.585343 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399442472 usec, L: 0
1028 03:00:02.585403 (+ 60us) write_op.cc:270] Start()
1028 03:00:02.585413 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399454164 usec, L: 0
1028 03:00:02.585472 (+ 59us) write_op.cc:270] Start()
1028 03:00:02.585482 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399455590 usec, L: 0
1028 03:00:02.585555 (+ 73us) write_op.cc:270] Start()
1028 03:00:02.585566 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399456244 usec, L: 0
1028 03:00:02.585649 (+ 83us) write_op.cc:270] Start()
1028 03:00:02.585664 (+ 15us) write_op.cc:276] Timestamp: P: 1761620399470283 usec, L: 0
1028 03:00:02.585746 (+ 82us) write_op.cc:270] Start()
1028 03:00:02.585759 (+ 13us) write_op.cc:276] Timestamp: P: 1761620399474987 usec, L: 0
1028 03:00:02.585841 (+ 82us) write_op.cc:270] Start()
1028 03:00:02.585853 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399483716 usec, L: 0
1028 03:00:02.585961 (+ 108us) write_op.cc:270] Start()
1028 03:00:02.585975 (+ 14us) write_op.cc:276] Timestamp: P: 1761620399485614 usec, L: 0
1028 03:00:02.586088 (+ 113us) write_op.cc:270] Start()
1028 03:00:02.586106 (+ 18us) write_op.cc:276] Timestamp: P: 1761620399486199 usec, L: 0
1028 03:00:02.586200 (+ 94us) write_op.cc:270] Start()
1028 03:00:02.586219 (+ 19us) write_op.cc:276] Timestamp: P: 1761620399486880 usec, L: 0
1028 03:00:02.586303 (+ 84us) write_op.cc:270] Start()
1028 03:00:02.586315 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399487351 usec, L: 0
1028 03:00:02.586394 (+ 79us) write_op.cc:270] Start()
1028 03:00:02.586405 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399496754 usec, L: 0
1028 03:00:02.586492 (+ 87us) write_op.cc:270] Start()
1028 03:00:02.586504 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399518475 usec, L: 0
1028 03:00:02.586588 (+ 84us) write_op.cc:270] Start()
1028 03:00:02.586614 (+ 26us) write_op.cc:276] Timestamp: P: 1761620399524025 usec, L: 0
1028 03:00:02.586702 (+ 88us) write_op.cc:270] Start()
1028 03:00:02.586715 (+ 13us) write_op.cc:276] Timestamp: P: 1761620399554236 usec, L: 0
1028 03:00:02.586798 (+ 83us) write_op.cc:270] Start()
1028 03:00:02.586815 (+ 17us) write_op.cc:276] Timestamp: P: 1761620399557667 usec, L: 0
1028 03:00:02.586915 (+ 100us) write_op.cc:270] Start()
1028 03:00:02.586930 (+ 15us) write_op.cc:276] Timestamp: P: 1761620399574817 usec, L: 0
1028 03:00:02.591386 (+ 4456us) write_op.cc:270] Start()
1028 03:00:02.591423 (+ 37us) write_op.cc:276] Timestamp: P: 1761620399600701 usec, L: 0
1028 03:00:02.591600 (+ 177us) write_op.cc:270] Start()
1028 03:00:02.591615 (+ 15us) write_op.cc:276] Timestamp: P: 1761620399613969 usec, L: 0
1028 03:00:02.591704 (+ 89us) write_op.cc:270] Start()
1028 03:00:02.591718 (+ 14us) write_op.cc:276] Timestamp: P: 1761620399614593 usec, L: 0
1028 03:00:02.591802 (+ 84us) write_op.cc:270] Start()
1028 03:00:02.591814 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399627256 usec, L: 0
1028 03:00:02.591874 (+ 60us) write_op.cc:270] Start()
1028 03:00:02.591884 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399628958 usec, L: 0
1028 03:00:02.591947 (+ 63us) write_op.cc:270] Start()
1028 03:00:02.591957 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399636731 usec, L: 0
1028 03:00:02.592030 (+ 73us) write_op.cc:270] Start()
1028 03:00:02.592041 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399637785 usec, L: 0
1028 03:00:02.592121 (+ 80us) write_op.cc:270] Start()
1028 03:00:02.592132 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399639856 usec, L: 0
1028 03:00:02.592199 (+ 67us) write_op.cc:270] Start()
1028 03:00:02.592210 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399653800 usec, L: 0
1028 03:00:02.592273 (+ 63us) write_op.cc:270] Start()
1028 03:00:02.592283 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399654529 usec, L: 0
1028 03:00:02.592348 (+ 65us) write_op.cc:270] Start()
1028 03:00:02.592358 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399704101 usec, L: 0
1028 03:00:02.592434 (+ 76us) write_op.cc:270] Start()
1028 03:00:02.592445 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399706249 usec, L: 0
1028 03:00:02.592527 (+ 82us) write_op.cc:270] Start()
1028 03:00:02.592538 (+ 11us) write_op.cc:276] Timestamp: P: 1761620399717071 usec, L: 0
1028 03:00:02.592609 (+ 71us) write_op.cc:270] Start()
1028 03:00:02.592634 (+ 25us) write_op.cc:276] Timestamp: P: 1761620399750017 usec, L: 0
1028 03:00:02.592701 (+ 67us) write_op.cc:270] Start()
1028 03:00:02.592713 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399759869 usec, L: 0
1028 03:00:02.592801 (+ 88us) write_op.cc:270] Start()
1028 03:00:02.592813 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399760859 usec, L: 0
1028 03:00:02.592900 (+ 87us) write_op.cc:270] Start()
1028 03:00:02.592915 (+ 15us) write_op.cc:276] Timestamp: P: 1761620399762250 usec, L: 0
1028 03:00:02.592987 (+ 72us) write_op.cc:270] Start()
1028 03:00:02.593007 (+ 20us) write_op.cc:276] Timestamp: P: 1761620399764809 usec, L: 0
1028 03:00:02.593071 (+ 64us) write_op.cc:270] Start()
1028 03:00:02.593084 (+ 13us) write_op.cc:276] Timestamp: P: 1761620399766186 usec, L: 0
1028 03:00:02.593170 (+ 86us) write_op.cc:270] Start()
1028 03:00:02.593187 (+ 17us) write_op.cc:276] Timestamp: P: 1761620399766840 usec, L: 0
1028 03:00:02.593274 (+ 87us) write_op.cc:270] Start()
1028 03:00:02.599206 (+ 5932us) write_op.cc:276] Timestamp: P: 1761620399809877 usec, L: 0
1028 03:00:02.600432 (+ 1226us) write_op.cc:270] Start()
1028 03:00:02.600459 (+ 27us) write_op.cc:276] Timestamp: P: 1761620399816392 usec, L: 0
1028 03:00:02.600571 (+ 112us) write_op.cc:270] Start()
1028 03:00:02.600586 (+ 15us) write_op.cc:276] Timestamp: P: 1761620399824905 usec, L: 0
1028 03:00:02.600712 (+ 126us) write_op.cc:270] Start()
1028 03:00:02.600726 (+ 14us) write_op.cc:276] Timestamp: P: 1761620399830207 usec, L: 0
1028 03:00:02.600804 (+ 78us) write_op.cc:270] Start()
1028 03:00:02.600819 (+ 15us) write_op.cc:276] Timestamp: P: 1761620399839638 usec, L: 0
1028 03:00:02.600913 (+ 94us) write_op.cc:270] Start()
1028 03:00:02.600927 (+ 14us) write_op.cc:276] Timestamp: P: 1761620399843181 usec, L: 0
1028 03:00:02.601007 (+ 80us) write_op.cc:270] Start()
1028 03:00:02.601019 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399853610 usec, L: 0
1028 03:00:02.601099 (+ 80us) write_op.cc:270] Start()
1028 03:00:02.601111 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399855558 usec, L: 0
1028 03:00:02.601203 (+ 92us) write_op.cc:270] Start()
1028 03:00:02.601216 (+ 13us) write_op.cc:276] Timestamp: P: 1761620399890462 usec, L: 0
1028 03:00:02.601294 (+ 78us) write_op.cc:270] Start()
1028 03:00:02.601307 (+ 13us) write_op.cc:276] Timestamp: P: 1761620399916092 usec, L: 0
1028 03:00:02.601387 (+ 80us) write_op.cc:270] Start()
1028 03:00:02.601400 (+ 13us) write_op.cc:276] Timestamp: P: 1761620399930101 usec, L: 0
1028 03:00:02.601500 (+ 100us) write_op.cc:270] Start()
1028 03:00:02.601519 (+ 19us) write_op.cc:276] Timestamp: P: 1761620399945346 usec, L: 0
1028 03:00:02.601604 (+ 85us) write_op.cc:270] Start()
1028 03:00:02.601616 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399946051 usec, L: 0
1028 03:00:02.601683 (+ 67us) write_op.cc:270] Start()
1028 03:00:02.601695 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399953499 usec, L: 0
1028 03:00:02.601772 (+ 77us) write_op.cc:270] Start()
1028 03:00:02.601784 (+ 12us) write_op.cc:276] Timestamp: P: 1761620399966693 usec, L: 0
1028 03:00:02.601856 (+ 72us) write_op.cc:270] Start()
1028 03:00:02.601866 (+ 10us) write_op.cc:276] Timestamp: P: 1761620399971983 usec, L: 0
1028 03:00:02.601926 (+ 60us) write_op.cc:270] Start()
1028 03:00:02.601936 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400070854 usec, L: 0
1028 03:00:02.602012 (+ 76us) write_op.cc:270] Start()
1028 03:00:02.602023 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400071554 usec, L: 0
1028 03:00:02.602092 (+ 69us) write_op.cc:270] Start()
1028 03:00:02.602103 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400071920 usec, L: 0
1028 03:00:02.602175 (+ 72us) write_op.cc:270] Start()
1028 03:00:02.602186 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400072221 usec, L: 0
1028 03:00:02.607633 (+ 5447us) write_op.cc:270] Start()
1028 03:00:02.607653 (+ 20us) write_op.cc:276] Timestamp: P: 1761620400072506 usec, L: 0
1028 03:00:02.607755 (+ 102us) write_op.cc:270] Start()
1028 03:00:02.607769 (+ 14us) write_op.cc:276] Timestamp: P: 1761620400072812 usec, L: 0
1028 03:00:02.607836 (+ 67us) write_op.cc:270] Start()
1028 03:00:02.607847 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400073130 usec, L: 0
1028 03:00:02.607942 (+ 95us) write_op.cc:270] Start()
1028 03:00:02.607953 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400073451 usec, L: 0
1028 03:00:02.608036 (+ 83us) write_op.cc:270] Start()
1028 03:00:02.608061 (+ 25us) write_op.cc:276] Timestamp: P: 1761620400073774 usec, L: 0
1028 03:00:02.608137 (+ 76us) write_op.cc:270] Start()
1028 03:00:02.608148 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400074203 usec, L: 0
1028 03:00:02.608225 (+ 77us) write_op.cc:270] Start()
1028 03:00:02.608236 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400102113 usec, L: 0
1028 03:00:02.608327 (+ 91us) write_op.cc:270] Start()
1028 03:00:02.608338 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400102513 usec, L: 0
1028 03:00:02.608403 (+ 65us) write_op.cc:270] Start()
1028 03:00:02.608418 (+ 15us) write_op.cc:276] Timestamp: P: 1761620400113745 usec, L: 0
1028 03:00:02.608479 (+ 61us) write_op.cc:270] Start()
1028 03:00:02.608489 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400120598 usec, L: 0
1028 03:00:02.608547 (+ 58us) write_op.cc:270] Start()
1028 03:00:02.608557 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400121016 usec, L: 0
1028 03:00:02.608632 (+ 75us) write_op.cc:270] Start()
1028 03:00:02.608643 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400121319 usec, L: 0
1028 03:00:02.608708 (+ 65us) write_op.cc:270] Start()
1028 03:00:02.608718 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400154820 usec, L: 0
1028 03:00:02.608799 (+ 81us) write_op.cc:270] Start()
1028 03:00:02.608811 (+ 12us) write_op.cc:276] Timestamp: P: 1761620400157597 usec, L: 0
1028 03:00:02.608873 (+ 62us) write_op.cc:270] Start()
1028 03:00:02.608884 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400181460 usec, L: 0
1028 03:00:02.608954 (+ 70us) write_op.cc:270] Start()
1028 03:00:02.608966 (+ 12us) write_op.cc:276] Timestamp: P: 1761620400182618 usec, L: 0
1028 03:00:02.614638 (+ 5672us) write_op.cc:270] Start()
1028 03:00:02.614658 (+ 20us) write_op.cc:276] Timestamp: P: 1761620400184872 usec, L: 0
1028 03:00:02.614753 (+ 95us) write_op.cc:270] Start()
1028 03:00:02.614763 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400207634 usec, L: 0
1028 03:00:02.614816 (+ 53us) write_op.cc:270] Start()
1028 03:00:02.614825 (+ 9us) write_op.cc:276] Timestamp: P: 1761620400212035 usec, L: 0
1028 03:00:02.614882 (+ 57us) write_op.cc:270] Start()
1028 03:00:02.614891 (+ 9us) write_op.cc:276] Timestamp: P: 1761620400224931 usec, L: 0
1028 03:00:02.614939 (+ 48us) write_op.cc:270] Start()
1028 03:00:02.614948 (+ 9us) write_op.cc:276] Timestamp: P: 1761620400233967 usec, L: 0
1028 03:00:02.615004 (+ 56us) write_op.cc:270] Start()
1028 03:00:02.615012 (+ 8us) write_op.cc:276] Timestamp: P: 1761620400284229 usec, L: 0
1028 03:00:02.615062 (+ 50us) write_op.cc:270] Start()
1028 03:00:02.615069 (+ 7us) write_op.cc:276] Timestamp: P: 1761620400323003 usec, L: 0
1028 03:00:02.615130 (+ 61us) write_op.cc:270] Start()
1028 03:00:02.615139 (+ 9us) write_op.cc:276] Timestamp: P: 1761620400326714 usec, L: 0
1028 03:00:02.615187 (+ 48us) write_op.cc:270] Start()
1028 03:00:02.615195 (+ 8us) write_op.cc:276] Timestamp: P: 1761620400337120 usec, L: 0
1028 03:00:02.615242 (+ 47us) write_op.cc:270] Start()
1028 03:00:02.615250 (+ 8us) write_op.cc:276] Timestamp: P: 1761620400337544 usec, L: 0
1028 03:00:02.615329 (+ 79us) write_op.cc:270] Start()
1028 03:00:02.615346 (+ 17us) write_op.cc:276] Timestamp: P: 1761620400337867 usec, L: 0
1028 03:00:02.615475 (+ 129us) write_op.cc:270] Start()
1028 03:00:02.615492 (+ 17us) write_op.cc:276] Timestamp: P: 1761620400338174 usec, L: 0
1028 03:00:02.615577 (+ 85us) write_op.cc:270] Start()
1028 03:00:02.615589 (+ 12us) write_op.cc:276] Timestamp: P: 1761620400338471 usec, L: 0
1028 03:00:02.615655 (+ 66us) write_op.cc:270] Start()
1028 03:00:02.615666 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400338771 usec, L: 0
1028 03:00:02.615725 (+ 59us) write_op.cc:270] Start()
1028 03:00:02.615735 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400339080 usec, L: 0
1028 03:00:02.615832 (+ 97us) write_op.cc:270] Start()
1028 03:00:02.615844 (+ 12us) write_op.cc:276] Timestamp: P: 1761620400341762 usec, L: 0
1028 03:00:02.615909 (+ 65us) write_op.cc:270] Start()
1028 03:00:02.615920 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400377006 usec, L: 0
1028 03:00:02.615991 (+ 71us) write_op.cc:270] Start()
1028 03:00:02.616011 (+ 20us) write_op.cc:276] Timestamp: P: 1761620400407095 usec, L: 0
1028 03:00:02.616075 (+ 64us) write_op.cc:270] Start()
1028 03:00:02.616086 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400408316 usec, L: 0
1028 03:00:02.616167 (+ 81us) write_op.cc:270] Start()
1028 03:00:02.616180 (+ 13us) write_op.cc:276] Timestamp: P: 1761620400409237 usec, L: 0
1028 03:00:02.616248 (+ 68us) write_op.cc:270] Start()
1028 03:00:02.616259 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400419814 usec, L: 0
1028 03:00:02.616321 (+ 62us) write_op.cc:270] Start()
1028 03:00:02.616332 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400420307 usec, L: 0
1028 03:00:02.616404 (+ 72us) write_op.cc:270] Start()
1028 03:00:02.616414 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400420713 usec, L: 0
1028 03:00:02.616486 (+ 72us) write_op.cc:270] Start()
1028 03:00:02.616497 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400421127 usec, L: 0
1028 03:00:02.616564 (+ 67us) write_op.cc:270] Start()
1028 03:00:02.616575 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400421492 usec, L: 0
1028 03:00:02.616644 (+ 69us) write_op.cc:270] Start()
1028 03:00:02.616654 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400427654 usec, L: 0
1028 03:00:02.616713 (+ 59us) write_op.cc:270] Start()
1028 03:00:02.616724 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400469419 usec, L: 0
1028 03:00:02.616793 (+ 69us) write_op.cc:270] Start()
1028 03:00:02.616804 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400470055 usec, L: 0
1028 03:00:02.616869 (+ 65us) write_op.cc:270] Start()
1028 03:00:02.616908 (+ 39us) write_op.cc:276] Timestamp: P: 1761620400470812 usec, L: 0
1028 03:00:02.617026 (+ 118us) write_op.cc:270] Start()
1028 03:00:02.617038 (+ 12us) write_op.cc:276] Timestamp: P: 1761620400476693 usec, L: 0
1028 03:00:02.617108 (+ 70us) write_op.cc:270] Start()
1028 03:00:02.617120 (+ 12us) write_op.cc:276] Timestamp: P: 1761620400495773 usec, L: 0
1028 03:00:02.617193 (+ 73us) write_op.cc:270] Start()
1028 03:00:02.617204 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400517138 usec, L: 0
1028 03:00:02.617269 (+ 65us) write_op.cc:270] Start()
1028 03:00:02.617279 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400529651 usec, L: 0
1028 03:00:02.617359 (+ 80us) write_op.cc:270] Start()
1028 03:00:02.617369 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400531022 usec, L: 0
1028 03:00:02.617432 (+ 63us) write_op.cc:270] Start()
1028 03:00:02.617443 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400533485 usec, L: 0
1028 03:00:02.617514 (+ 71us) write_op.cc:270] Start()
1028 03:00:02.617525 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400586194 usec, L: 0
1028 03:00:02.617590 (+ 65us) write_op.cc:270] Start()
1028 03:00:02.617600 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400620011 usec, L: 0
1028 03:00:02.617660 (+ 60us) write_op.cc:270] Start()
1028 03:00:02.617670 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400627784 usec, L: 0
1028 03:00:02.617734 (+ 64us) write_op.cc:270] Start()
1028 03:00:02.617744 (+ 10us) write_op.cc:276] Timestamp: P: 1761620400662995 usec, L: 0
1028 03:00:02.617818 (+ 74us) write_op.cc:270] Start()
1028 03:00:02.617835 (+ 17us) write_op.cc:276] Timestamp: P: 1761620400669635 usec, L: 0
1028 03:00:02.617911 (+ 76us) write_op.cc:270] Start()
1028 03:00:02.617922 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400679971 usec, L: 0
1028 03:00:02.617985 (+ 63us) write_op.cc:270] Start()
1028 03:00:02.618003 (+ 18us) write_op.cc:276] Timestamp: P: 1761620400680554 usec, L: 0
1028 03:00:02.618068 (+ 65us) write_op.cc:270] Start()
1028 03:00:02.618079 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400680956 usec, L: 0
1028 03:00:02.618152 (+ 73us) write_op.cc:270] Start()
1028 03:00:02.618163 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400687741 usec, L: 0
1028 03:00:02.618239 (+ 76us) write_op.cc:270] Start()
1028 03:00:02.618250 (+ 11us) write_op.cc:276] Timestamp: P: 1761620400693705 usec, L: 0
1028 03:00:02.622268 (+ 4018us) log.cc:844] Serialized 116968 byte log entry
1028 03:00:02.622608 (+ 340us) raft_consensus.cc:1645] Marking committed up to 1115
1028 03:00:02.625798 (+ 3190us) raft_consensus.cc:1697] Filling consensus response to leader.
1028 03:00:02.625878 (+ 80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
1028 03:00:02.776801 (+150923us) raft_consensus.cc:1685] finished
1028 03:00:02.776807 (+ 6us) raft_consensus.cc:1691] UpdateReplicas() finished
1028 03:00:02.776926 (+ 119us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1028 03:00:02.596491 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1028 03:00:02.596619 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
1028 03:00:02.596623 (+ 4us) write_op.cc:435] Acquired schema lock
1028 03:00:02.596625 (+ 2us) tablet.cc:660] Decoding operations
1028 03:00:02.596732 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
1028 03:00:02.596744 (+ 12us) write_op.cc:641] Partition lock acquired for write op
1028 03:00:02.596747 (+ 3us) tablet.cc:683] Acquiring locks for 2 operations
1028 03:00:02.596804 (+ 57us) tablet.cc:699] Row locks acquired
1028 03:00:02.596805 (+ 1us) write_op.cc:260] PREPARE: finished
1028 03:00:02.622751 (+ 25946us) op_driver.cc:464] REPLICATION: finished
1028 03:00:02.740661 (+117910us) write_op.cc:301] APPLY: starting
1028 03:00:02.740687 (+ 26us) tablet.cc:1365] starting BulkCheckPresence
1028 03:00:02.740966 (+ 279us) tablet.cc:1368] finished BulkCheckPresence
1028 03:00:02.740968 (+ 2us) tablet.cc:1370] starting ApplyRowOperation cycle
1028 03:00:02.741068 (+ 100us) tablet.cc:1381] finished ApplyRowOperation cycle
1028 03:00:02.741210 (+ 142us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1028 03:00:02.741267 (+ 57us) write_op.cc:312] APPLY: finished
1028 03:00:02.741379 (+ 112us) log.cc:844] Serialized 29 byte log entry
1028
I20251028 03:00:02.837397 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5fe5aa8ebc9c4ee6959d74c8c6050480
I20251028 03:00:02.838328 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 88cb16241e364f56b251d88079936c7b
I20251028 03:00:02.838603 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 88cb16241e364f56b251d88079936c7b deleted and replaced by tablet c99a9bd6c364463d913cfe758f3a981c
W20251028 03:00:02.840041 7967 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9da148c6c854499aa0bd234ba2b0a82d consensus state has no leader
I20251028 03:00:02.844141 7961 catalog_manager.cc:5649] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.844628 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 338cf9d02142436f97ccd44313cc8199 (table default.Table_77a37bbdfa4f41458e6b50fb213850bc [id=5f61b3c12ed647968fc2932aa24a13f3]). Expected version 2 got 0
I20251028 03:00:02.845085 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet e3b33c4d189247a89878b6a8e1fcbac0 (table default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]). Expected version 1 got 0
I20251028 03:00:02.845301 7961 catalog_manager.cc:5649] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.845644 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 2bf100407b034626b4a9f7e2340f9cf0 (table default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262]). Expected version 2 got 0
I20251028 03:00:02.845780 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet ad926e9ee0044c62a7abbfca959a4f56 (table default.Table_6ad31a79d9f043628bdac1dd2d4c263c [id=4570d7ddf33c447fb40d6411ea47e52a]). Expected version 1 got 0
I20251028 03:00:02.845970 7961 catalog_manager.cc:5649] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.846215 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 8665d2e933fa423d81a1b32f2ce8629b (table default.Table_139255bc4fb74fae8fb86563b54826bd [id=dc3721126b09466ab5efdb8bd4b46d89]). Expected version 1 got 0
I20251028 03:00:02.850545 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.850735 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 88cb16241e364f56b251d88079936c7b with delete_type TABLET_DATA_DELETED (replaced by tablet c99a9bd6c364463d913cfe758f3a981c) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.851536 8112 tablet_service.cc:1505] Processing CreateTablet for tablet ad90c05da01e44aab6a978bb0d9e9674 (DEFAULT_TABLE table=default.Table_f43f1b75476441f294e6ca3839d1591f [id=524a44cef05242e2b97b87da0f78e537]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.851677 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 8e408c2f484b4f81ba8ef216dc64c87a (DEFAULT_TABLE table=default.Table_aadf4f11e0134fc78dff262c2726bc8a [id=a72c2ec6949f4e1d9bcff30e872fdbcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.851821 8118 tablet_service.cc:1505] Processing CreateTablet for tablet e972e2d157134594938bfb0719c477ae (DEFAULT_TABLE table=default.Table_c8239b64a71949bb8553a15b778102b5 [id=071c10a40435470ab26285fe1b340e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.851650 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 094eb6139ee64077a23844be7e899d06 (DEFAULT_TABLE table=default.Table_149a395a74c243f29f343d30b3cf6bb6 [id=bdae370d10c143b9a703a63dfccbfe53]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.851956 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 524b582bd3bc46be92393ee933c860c7 (DEFAULT_TABLE table=default.Table_555cd9726353408bbafada707d406c8d [id=d1d6a42ea4484ec1984883c8ede5e6cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.852010 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 427ae7e31e81447ba6f8c8b8ccf4761c (DEFAULT_TABLE table=default.Table_8a28a3e233bb4011b36e95509d454cc6 [id=40afefe7bc06446c98bb48eb0d7eb508]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.852145 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 496fe9d4a4a14bd49d97d8ff2e63e146 (DEFAULT_TABLE table=default.Table_1b11f8e3f61d4362a6984c4ac6c9ff8d [id=e72cf6873db8453d8bbf90b2d61b1798]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.852344 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 49cc6983acdb430288ab0bb65145998e (DEFAULT_TABLE table=default.Table_642de38725d443f7861a2230d095c55c [id=fa61cd3a65e84df5b895bbda4d14f5ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.852370 8115 tablet_service.cc:1505] Processing CreateTablet for tablet 09e13b6d50b14e1dadf5a67bc42337a0 (DEFAULT_TABLE table=default.Table_5767228cc6d84aa0b3d29bee0c6cb362 [id=6a6e59f6170043e8a17e1a949371088b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.852557 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 6d238a79585e481e97efee4ffd082421 (DEFAULT_TABLE table=default.Table_cd4df8be533e472bafeb04d0aa635832 [id=1b512207483d41adb36d9d76c3b560f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.852766 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 362353a0ae094c53a81251bcc8ce09f7 (DEFAULT_TABLE table=default.Table_ce2d5682714a4f6382b57097402f7f55 [id=29f9dce9a37f4cbca05d9192287b335f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.852845 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 6f3d365e4a834442aece57fe87b64bc6 (DEFAULT_TABLE table=default.Table_49fb24aa7f12420fae313eaa20ac98b0 [id=72279b27b10e48b28b4b914508ad0169]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.853433 8124 tablet_service.cc:1505] Processing CreateTablet for tablet fcc757c38ac6492e9697bcdd053364c6 (DEFAULT_TABLE table=default.Table_e9c0af79b3c44383883fa3c5ecfa256d [id=10af3345ea1f43668a7e444f6aa5ebaa]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 03:00:02.817746 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 36cfa812c550475385980578108fb6be because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36cfa812c550475385980578108fb6be already in progress: creating tablet
I20251028 03:00:02.857178 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet fd8c04efd8c2456b9c7643641c7fbc0d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.857365 8551 tablet_replica.cc:333] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:02.857528 8551 raft_consensus.cc:2243] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:02.857671 8551 pending_rounds.cc:70] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 03:00:02.857708 8551 pending_rounds.cc:77] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597168792682496 op_type: NO_OP noop_request { }
I20251028 03:00:02.857782 8551 raft_consensus.cc:2889] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:02.857846 8551 raft_consensus.cc:2272] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.858911 8551 ts_tablet_manager.cc:1916] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.859741 10192 tablet.cc:1722] T aa36c6870c1b4a459f0a41c84855a76e P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:02.861492 10253 raft_consensus.cc:493] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.861578 8551 ts_tablet_manager.cc:1929] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:02.861652 8551 log.cc:1199] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/fd8c04efd8c2456b9c7643641c7fbc0d
I20251028 03:00:02.861605 10253 raft_consensus.cc:515] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.861936 8551 ts_tablet_manager.cc:1950] T fd8c04efd8c2456b9c7643641c7fbc0d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:02.861990 10253 leader_election.cc:290] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.863055 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e3b33c4d189247a89878b6a8e1fcbac0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.863348 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e3b33c4d189247a89878b6a8e1fcbac0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:02.863471 8420 raft_consensus.cc:2468] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:02.863530 8273 raft_consensus.cc:3060] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:02.863849 8083 leader_election.cc:343] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.864260 8416 raft_consensus.cc:1275] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:02.864667 8273 raft_consensus.cc:1275] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:02.865120 8083 leader_election.cc:304] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:02.865271 10246 consensus_queue.cc:1048] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.865291 9788 consensus_queue.cc:1048] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.865617 9788 raft_consensus.cc:2804] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:02.865672 9788 raft_consensus.cc:493] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:02.865715 9788 raft_consensus.cc:3060] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.866431 10187 tablet.cc:1722] T aa36c6870c1b4a459f0a41c84855a76e P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:02.866726 9788 raft_consensus.cc:515] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.867082 9788 leader_election.cc:290] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.867175 9788 consensus_queue.cc:1048] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.868300 8275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e3b33c4d189247a89878b6a8e1fcbac0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:02.868502 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e3b33c4d189247a89878b6a8e1fcbac0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:02.868615 8416 raft_consensus.cc:3060] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:02.869618 8416 raft_consensus.cc:2468] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:02.869997 8083 leader_election.cc:304] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:02.870162 10253 raft_consensus.cc:2804] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:02.870225 10253 raft_consensus.cc:697] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:02.870357 10253 consensus_queue.cc:237] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:02.871312 8083 leader_election.cc:343] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.871846 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.872077 8551 tablet_replica.cc:333] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:02.872409 8551 raft_consensus.cc:2243] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.872619 8551 raft_consensus.cc:2272] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.872718 9909 tablet.cc:1722] T aa36c6870c1b4a459f0a41c84855a76e P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:02.873338 10242 raft_consensus.cc:493] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.873441 10242 raft_consensus.cc:515] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.873816 10242 leader_election.cc:290] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:02.875133 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 already in progress: creating tablet
W20251028 03:00:02.875643 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 88cb16241e364f56b251d88079936c7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 88cb16241e364f56b251d88079936c7b already in progress: creating tablet
I20251028 03:00:02.875780 8275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee2fd98744ff44028f36bb55e6eba51e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:02.876216 8356 leader_election.cc:343] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.876425 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee2fd98744ff44028f36bb55e6eba51e" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:02.876865 8354 leader_election.cc:343] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.876940 8354 leader_election.cc:304] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:02.877182 10242 raft_consensus.cc:2749] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.878141 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 12d103e9b5024a87aa232c26e0c51ab9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.878866 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.879411 8552 tablet_replica.cc:333] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:02.879428 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.879581 8552 raft_consensus.cc:2243] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.879720 8552 raft_consensus.cc:2272] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.879933 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:02 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.880304 8552 ts_tablet_manager.cc:1916] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.880623 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 88cb16241e364f56b251d88079936c7b with delete_type TABLET_DATA_DELETED (replaced by tablet c99a9bd6c364463d913cfe758f3a981c) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.880955 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 88cb16241e364f56b251d88079936c7b with delete_type TABLET_DATA_DELETED (replaced by tablet c99a9bd6c364463d913cfe758f3a981c) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.881943 8258 tablet_service.cc:1505] Processing CreateTablet for tablet ad90c05da01e44aab6a978bb0d9e9674 (DEFAULT_TABLE table=default.Table_f43f1b75476441f294e6ca3839d1591f [id=524a44cef05242e2b97b87da0f78e537]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.882229 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad90c05da01e44aab6a978bb0d9e9674. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.882256 8392 tablet_service.cc:1505] Processing CreateTablet for tablet ad90c05da01e44aab6a978bb0d9e9674 (DEFAULT_TABLE table=default.Table_f43f1b75476441f294e6ca3839d1591f [id=524a44cef05242e2b97b87da0f78e537]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.882532 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad90c05da01e44aab6a978bb0d9e9674. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.882984 8552 ts_tablet_manager.cc:1929] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.883052 8552 log.cc:1199] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9277a4ffd0ca4bf7afd29e1c6ab33bc9
I20251028 03:00:02.883253 8552 ts_tablet_manager.cc:1950] T 9277a4ffd0ca4bf7afd29e1c6ab33bc9 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:02.885910 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 427ae7e31e81447ba6f8c8b8ccf4761c (DEFAULT_TABLE table=default.Table_8a28a3e233bb4011b36e95509d454cc6 [id=40afefe7bc06446c98bb48eb0d7eb508]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.886173 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 427ae7e31e81447ba6f8c8b8ccf4761c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.886830 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 094eb6139ee64077a23844be7e899d06 (DEFAULT_TABLE table=default.Table_149a395a74c243f29f343d30b3cf6bb6 [id=bdae370d10c143b9a703a63dfccbfe53]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.887051 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094eb6139ee64077a23844be7e899d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.887485 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 094eb6139ee64077a23844be7e899d06 (DEFAULT_TABLE table=default.Table_149a395a74c243f29f343d30b3cf6bb6 [id=bdae370d10c143b9a703a63dfccbfe53]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.887775 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094eb6139ee64077a23844be7e899d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.889896 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet f422030a6a4042009efc7fcfb21a6bca (table default.Table_a01151cafd944efcbc12e5f654a7a05a [id=fdd6ecd598f240948882d710fd485da2]). Expected version 2 got 0
I20251028 03:00:02.890084 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 338cf9d02142436f97ccd44313cc8199 (table default.Table_77a37bbdfa4f41458e6b50fb213850bc [id=5f61b3c12ed647968fc2932aa24a13f3]). Expected version 2 got 0
I20251028 03:00:02.890193 7962 catalog_manager.cc:5649] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 03:00:02.890502 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 5c1994484abf4b6fbbc3f47527558cdb (table default.Table_0d5bef57eadd45538258da2301f0cd31 [id=b0407c8f78a84245a954c67e888da3ec]). Expected version 5 got 0
I20251028 03:00:02.890604 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 2bf100407b034626b4a9f7e2340f9cf0 (table default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262]). Expected version 2 got 0
I20251028 03:00:02.890856 8392 tablet_service.cc:1505] Processing CreateTablet for tablet 427ae7e31e81447ba6f8c8b8ccf4761c (DEFAULT_TABLE table=default.Table_8a28a3e233bb4011b36e95509d454cc6 [id=40afefe7bc06446c98bb48eb0d7eb508]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.891427 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 427ae7e31e81447ba6f8c8b8ccf4761c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.892765 7964 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_49ace696414843d480acafa81bb4be89 [id=7e087d7232254c0e87aabe25be6eb25d] alter complete (version 1)
I20251028 03:00:02.897823 10242 raft_consensus.cc:493] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.897938 10242 raft_consensus.cc:515] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.898160 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 8e408c2f484b4f81ba8ef216dc64c87a (DEFAULT_TABLE table=default.Table_aadf4f11e0134fc78dff262c2726bc8a [id=a72c2ec6949f4e1d9bcff30e872fdbcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.898393 10242 leader_election.cc:290] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.898454 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e408c2f484b4f81ba8ef216dc64c87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.898689 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 8e408c2f484b4f81ba8ef216dc64c87a (DEFAULT_TABLE table=default.Table_aadf4f11e0134fc78dff262c2726bc8a [id=a72c2ec6949f4e1d9bcff30e872fdbcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.898975 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e408c2f484b4f81ba8ef216dc64c87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.899197 8551 ts_tablet_manager.cc:1916] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.900959 8258 tablet_service.cc:1505] Processing CreateTablet for tablet e972e2d157134594938bfb0719c477ae (DEFAULT_TABLE table=default.Table_c8239b64a71949bb8553a15b778102b5 [id=071c10a40435470ab26285fe1b340e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.901691 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e972e2d157134594938bfb0719c477ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.901787 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "597d792ed13941a2a895b0dc305614f1" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:02.902245 8354 leader_election.cc:343] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.902902 8275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "597d792ed13941a2a895b0dc305614f1" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:02.903079 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet fd8c04efd8c2456b9c7643641c7fbc0d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:02.903975 8356 leader_election.cc:343] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.904052 8356 leader_election.cc:304] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:02.904271 10242 raft_consensus.cc:2749] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.906064 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 09e13b6d50b14e1dadf5a67bc42337a0 (DEFAULT_TABLE table=default.Table_5767228cc6d84aa0b3d29bee0c6cb362 [id=6a6e59f6170043e8a17e1a949371088b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.905959 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 496fe9d4a4a14bd49d97d8ff2e63e146 (DEFAULT_TABLE table=default.Table_1b11f8e3f61d4362a6984c4ac6c9ff8d [id=e72cf6873db8453d8bbf90b2d61b1798]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.906364 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09e13b6d50b14e1dadf5a67bc42337a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.906519 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 496fe9d4a4a14bd49d97d8ff2e63e146. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.906862 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 09e13b6d50b14e1dadf5a67bc42337a0 (DEFAULT_TABLE table=default.Table_5767228cc6d84aa0b3d29bee0c6cb362 [id=6a6e59f6170043e8a17e1a949371088b]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 03:00:02.907094 8083 consensus_peers.cc:597] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:02.907146 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09e13b6d50b14e1dadf5a67bc42337a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.908083 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c41d49f9eba40a48bd6900671297db9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.909157 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet b7d3062426434c339be4a745decae729 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:02.909827 8550 tablet_replica.cc:333] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:02.910000 8550 raft_consensus.cc:2243] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:02.910168 8550 pending_rounds.cc:70] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 03:00:02.910202 8550 pending_rounds.cc:77] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597169856446464 op_type: NO_OP noop_request { }
I20251028 03:00:02.910267 8550 raft_consensus.cc:2889] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:02.910316 8550 raft_consensus.cc:2272] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.911006 8550 ts_tablet_manager.cc:1916] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.906867 8416 raft_consensus.cc:1275] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Refusing update from remote peer fbd2afb4894c4e7a8520cc09f8593aa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251028 03:00:02.912427 10335 consensus_queue.cc:1048] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.913395 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 496fe9d4a4a14bd49d97d8ff2e63e146 (DEFAULT_TABLE table=default.Table_1b11f8e3f61d4362a6984c4ac6c9ff8d [id=e72cf6873db8453d8bbf90b2d61b1798]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 03:00:02.916779 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet fd8c04efd8c2456b9c7643641c7fbc0d because the tablet was not found. No further retry: Not found: Tablet not found: fd8c04efd8c2456b9c7643641c7fbc0d
I20251028 03:00:02.913726 10242 consensus_queue.cc:1048] T e9a28ff8c0ee46cb89ba7e28de2cce45 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.919093 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 496fe9d4a4a14bd49d97d8ff2e63e146. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.926792 10192 tablet.cc:1722] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 03:00:02.930361 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 524b582bd3bc46be92393ee933c860c7 (DEFAULT_TABLE table=default.Table_555cd9726353408bbafada707d406c8d [id=d1d6a42ea4484ec1984883c8ede5e6cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.930934 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 6d238a79585e481e97efee4ffd082421 (DEFAULT_TABLE table=default.Table_cd4df8be533e472bafeb04d0aa635832 [id=1b512207483d41adb36d9d76c3b560f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.931025 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 6d238a79585e481e97efee4ffd082421 (DEFAULT_TABLE table=default.Table_cd4df8be533e472bafeb04d0aa635832 [id=1b512207483d41adb36d9d76c3b560f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.932154 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 49cc6983acdb430288ab0bb65145998e (DEFAULT_TABLE table=default.Table_642de38725d443f7861a2230d095c55c [id=fa61cd3a65e84df5b895bbda4d14f5ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.932289 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 49cc6983acdb430288ab0bb65145998e (DEFAULT_TABLE table=default.Table_642de38725d443f7861a2230d095c55c [id=fa61cd3a65e84df5b895bbda4d14f5ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.932605 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 362353a0ae094c53a81251bcc8ce09f7 (DEFAULT_TABLE table=default.Table_ce2d5682714a4f6382b57097402f7f55 [id=29f9dce9a37f4cbca05d9192287b335f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.932986 8390 tablet_service.cc:1505] Processing CreateTablet for tablet 362353a0ae094c53a81251bcc8ce09f7 (DEFAULT_TABLE table=default.Table_ce2d5682714a4f6382b57097402f7f55 [id=29f9dce9a37f4cbca05d9192287b335f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.933396 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 6f3d365e4a834442aece57fe87b64bc6 (DEFAULT_TABLE table=default.Table_49fb24aa7f12420fae313eaa20ac98b0 [id=72279b27b10e48b28b4b914508ad0169]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.933624 8389 tablet_service.cc:1505] Processing CreateTablet for tablet 6f3d365e4a834442aece57fe87b64bc6 (DEFAULT_TABLE table=default.Table_49fb24aa7f12420fae313eaa20ac98b0 [id=72279b27b10e48b28b4b914508ad0169]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.933972 8252 tablet_service.cc:1505] Processing CreateTablet for tablet fcc757c38ac6492e9697bcdd053364c6 (DEFAULT_TABLE table=default.Table_e9c0af79b3c44383883fa3c5ecfa256d [id=10af3345ea1f43668a7e444f6aa5ebaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:02.937896 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet f422030a6a4042009efc7fcfb21a6bca (table default.Table_a01151cafd944efcbc12e5f654a7a05a [id=fdd6ecd598f240948882d710fd485da2]). Expected version 2 got 0
I20251028 03:00:02.938433 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 9a81841c22ae49c0a17931d90407accc (table default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]). Expected version 1 got 0
I20251028 03:00:02.938807 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 5c1994484abf4b6fbbc3f47527558cdb (table default.Table_0d5bef57eadd45538258da2301f0cd31 [id=b0407c8f78a84245a954c67e888da3ec]). Expected version 5 got 0
I20251028 03:00:02.939090 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (table default.Table_93b5aac358764b869e8589ccc023ea0e [id=674f76f9afed4f3f9e2c165e2479d938]). Expected version 1 got 0
I20251028 03:00:02.943733 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49cc6983acdb430288ab0bb65145998e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.943909 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d238a79585e481e97efee4ffd082421. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.944166 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362353a0ae094c53a81251bcc8ce09f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.944388 8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3d365e4a834442aece57fe87b64bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.944442 9190 raft_consensus.cc:493] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.944514 9190 raft_consensus.cc:515] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.944909 9190 leader_election.cc:290] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:02.945508 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0dad91f926648c781d1582733d6ae95" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:02.945917 8354 leader_election.cc:343] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.945960 8275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0dad91f926648c781d1582733d6ae95" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:02.946424 8356 leader_election.cc:343] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:02.946484 8356 leader_election.cc:304] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:02.946609 9190 raft_consensus.cc:2749] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.951819 8389 tablet_service.cc:1552] Processing DeleteTablet for tablet f39ed19223fa4268a01aaf4c7762d86e with delete_type TABLET_DATA_DELETED (replaced by tablet b6a2791098934a9c8aa21d8db88f2c9e) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.952301 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet fd8c04efd8c2456b9c7643641c7fbc0d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.952646 8390 tablet_service.cc:1552] Processing DeleteTablet for tablet b7d3062426434c339be4a745decae729 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.952718 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet 9621e42e6a324ecda50fe83c0c53119c with delete_type TABLET_DATA_DELETED (replaced by tablet bedb4fd3b46f4453b194cfaea0c3c701) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.953276 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bc54972260747f3af0c18d05c6a3442 with delete_type TABLET_DATA_DELETED (replaced by tablet 9d438110cfec42a991b6c40b0881cc62) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.953653 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet 2ff112c66f1a4cf9b232e6bb665755f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.954039 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet 6599ec60c3004e349e7e576e4d13c4fe with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.954375 8253 tablet_service.cc:1552] Processing DeleteTablet for tablet 12d103e9b5024a87aa232c26e0c51ab9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:02.954747 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dab70ca2d9f4b5fba338cb337b0b396 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.955098 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 597d792ed13941a2a895b0dc305614f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.955586 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c41d49f9eba40a48bd6900671297db9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:02.955703 8112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad90c05da01e44aab6a978bb0d9e9674. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.955868 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d238a79585e481e97efee4ffd082421. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.956033 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e408c2f484b4f81ba8ef216dc64c87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.956621 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362353a0ae094c53a81251bcc8ce09f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.957928 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcc757c38ac6492e9697bcdd053364c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.958058 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 496fe9d4a4a14bd49d97d8ff2e63e146. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.958212 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 524b582bd3bc46be92393ee933c860c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.958340 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09e13b6d50b14e1dadf5a67bc42337a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.959399 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3d365e4a834442aece57fe87b64bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.960695 8550 ts_tablet_manager.cc:1929] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:02.960855 8550 log.cc:1199] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/1c41d49f9eba40a48bd6900671297db9
I20251028 03:00:02.961210 8550 ts_tablet_manager.cc:1950] T 1c41d49f9eba40a48bd6900671297db9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:02.961385 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 094eb6139ee64077a23844be7e899d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.961509 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e972e2d157134594938bfb0719c477ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.964202 8551 ts_tablet_manager.cc:1929] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.964288 8551 log.cc:1199] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/6599ec60c3004e349e7e576e4d13c4fe
I20251028 03:00:02.964502 8551 ts_tablet_manager.cc:1950] T 6599ec60c3004e349e7e576e4d13c4fe P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:02.966995 9909 tablet.cc:1722] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 03:00:02.969542 10242 consensus_queue.cc:1048] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:02.972282 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49cc6983acdb430288ab0bb65145998e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.975417 7964 catalog_manager.cc:5649] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:02.975669 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet e3b33c4d189247a89878b6a8e1fcbac0 (table default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]). Expected version 1 got 0
I20251028 03:00:02.975829 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 9a81841c22ae49c0a17931d90407accc (table default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]). Expected version 1 got 0
I20251028 03:00:02.976378 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 427ae7e31e81447ba6f8c8b8ccf4761c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:02.982340 8551 tablet_replica.cc:333] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:02.982546 8551 raft_consensus.cc:2243] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.982681 8551 raft_consensus.cc:2272] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.983107 9595 raft_consensus.cc:493] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:02.983217 8551 ts_tablet_manager.cc:1916] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:02.983197 9595 raft_consensus.cc:515] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:02.983603 9595 leader_election.cc:290] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:02.987751 8551 ts_tablet_manager.cc:1929] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.987937 8551 log.cc:1199] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/12d103e9b5024a87aa232c26e0c51ab9
I20251028 03:00:02.988428 8551 ts_tablet_manager.cc:1950] T 12d103e9b5024a87aa232c26e0c51ab9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 03:00:02.988492 8220 leader_election.cc:343] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 9621e42e6a324ecda50fe83c0c53119c
I20251028 03:00:02.990278 8551 tablet_replica.cc:333] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:02.990551 8551 raft_consensus.cc:2243] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:02.990764 8551 raft_consensus.cc:2272] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:02.993285 8551 ts_tablet_manager.cc:1916] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:02.994956 8218 leader_election.cc:343] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 9621e42e6a324ecda50fe83c0c53119c
I20251028 03:00:02.995050 8218 leader_election.cc:304] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:02.996471 9595 raft_consensus.cc:2749] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:02.999153 8551 ts_tablet_manager.cc:1929] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:02.999275 8551 log.cc:1199] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/f39ed19223fa4268a01aaf4c7762d86e
I20251028 03:00:02.999563 8551 ts_tablet_manager.cc:1950] T f39ed19223fa4268a01aaf4c7762d86e P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.002512 8551 tablet_replica.cc:333] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.002708 8551 raft_consensus.cc:2243] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:03.002885 8551 pending_rounds.cc:70] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 03:00:03.002943 8551 pending_rounds.cc:77] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597169546207232 op_type: NO_OP noop_request { }
I20251028 03:00:03.003046 8551 raft_consensus.cc:2889] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:03.003126 8551 raft_consensus.cc:2272] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.005367 9190 raft_consensus.cc:493] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.005457 9190 raft_consensus.cc:515] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.005900 9190 leader_election.cc:290] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.006116 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba64524142934af08658e434e7fb08ed" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:03.006590 8551 ts_tablet_manager.cc:1916] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:03.006666 8354 leader_election.cc:343] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.007311 9190 consensus_queue.cc:1048] T 01d609dc2fea4c79adf0953dc98ad0af P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:03.007863 9190 consensus_queue.cc:1048] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:03.009037 8551 ts_tablet_manager.cc:1929] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:03.009469 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ba64524142934af08658e434e7fb08ed" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.013391 8551 log.cc:1199] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/b7d3062426434c339be4a745decae729
I20251028 03:00:03.013772 8551 ts_tablet_manager.cc:1950] T b7d3062426434c339be4a745decae729 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.016646 8551 tablet_replica.cc:333] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.017911 8551 raft_consensus.cc:2243] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.019317 8551 raft_consensus.cc:2272] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:03.019675 8356 leader_election.cc:343] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.019773 8356 leader_election.cc:304] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.020010 9190 raft_consensus.cc:2749] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.020284 8551 ts_tablet_manager.cc:1916] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.022053 10242 consensus_queue.cc:1048] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:03.028132 8551 ts_tablet_manager.cc:1929] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.028317 8551 log.cc:1199] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8dab70ca2d9f4b5fba338cb337b0b396
I20251028 03:00:03.028569 8551 ts_tablet_manager.cc:1950] T 8dab70ca2d9f4b5fba338cb337b0b396 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.029237 8551 tablet_replica.cc:333] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.029400 8551 raft_consensus.cc:2243] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.029769 8551 raft_consensus.cc:2272] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.030292 8551 ts_tablet_manager.cc:1916] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.032451 8551 ts_tablet_manager.cc:1929] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.032461 8544 tablet_bootstrap.cc:492] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.032562 8544 ts_tablet_manager.cc:1403] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.291s user 0.002s sys 0.000s
I20251028 03:00:03.032603 8551 log.cc:1199] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/597d792ed13941a2a895b0dc305614f1
I20251028 03:00:03.033038 8545 tablet_bootstrap.cc:492] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.033030 8544 raft_consensus.cc:359] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.033125 8545 ts_tablet_manager.cc:1403] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.274s user 0.002s sys 0.000s
I20251028 03:00:03.033123 8544 raft_consensus.cc:385] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.033174 8544 raft_consensus.cc:740] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.033280 8544 consensus_queue.cc:260] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.033581 8544 ts_tablet_manager.cc:1434] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:03.033551 8545 raft_consensus.cc:359] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.033641 8545 raft_consensus.cc:385] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.033674 8545 raft_consensus.cc:740] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.033777 8545 consensus_queue.cc:260] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.034915 8546 tablet_bootstrap.cc:492] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.035014 8546 ts_tablet_manager.cc:1403] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.307s user 0.003s sys 0.000s
I20251028 03:00:03.035411 8545 ts_tablet_manager.cc:1434] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:03.035595 8545 tablet_bootstrap.cc:492] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.035549 8546 raft_consensus.cc:359] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.035682 8546 raft_consensus.cc:385] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.035722 8546 raft_consensus.cc:740] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.035835 8546 consensus_queue.cc:260] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.036105 8546 ts_tablet_manager.cc:1434] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.036630 8545 tablet_bootstrap.cc:654] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.038015 8551 ts_tablet_manager.cc:1950] T 597d792ed13941a2a895b0dc305614f1 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.038707 8551 tablet_replica.cc:333] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.039469 8553 raft_consensus.cc:493] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.039672 8553 raft_consensus.cc:515] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.040378 8551 raft_consensus.cc:2243] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.040531 8551 raft_consensus.cc:2272] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.040555 9595 raft_consensus.cc:493] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.040654 9595 raft_consensus.cc:515] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.041035 9595 leader_election.cc:290] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:03.041428 8544 tablet_bootstrap.cc:492] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.041576 8551 ts_tablet_manager.cc:1916] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.042315 8553 leader_election.cc:290] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:03.042584 8544 tablet_bootstrap.cc:654] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:03.042937 8220 leader_election.cc:343] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: b7d3062426434c339be4a745decae729
W20251028 03:00:03.043462 8220 leader_election.cc:343] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: ab1ec08da9f64109a3fe9400fe4e366c
I20251028 03:00:03.043907 8551 ts_tablet_manager.cc:1929] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.044536 8545 tablet_bootstrap.cc:492] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.044651 8545 ts_tablet_manager.cc:1403] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 03:00:03.045141 8545 raft_consensus.cc:359] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.045238 8545 raft_consensus.cc:385] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.045274 8545 raft_consensus.cc:740] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.045393 8545 consensus_queue.cc:260] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.045678 8545 ts_tablet_manager.cc:1434] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.045868 8545 tablet_bootstrap.cc:492] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.046995 8545 tablet_bootstrap.cc:654] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.049093 8551 log.cc:1199] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/1c41d49f9eba40a48bd6900671297db9
I20251028 03:00:03.049425 8551 ts_tablet_manager.cc:1950] T 1c41d49f9eba40a48bd6900671297db9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.050679 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "b7d3062426434c339be4a745decae729" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:03.050860 8146 raft_consensus.cc:2393] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 03:00:03.051524 8218 leader_election.cc:304] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 03:00:03.052042 8218 leader_election.cc:343] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: ab1ec08da9f64109a3fe9400fe4e366c
I20251028 03:00:03.052115 8218 leader_election.cc:304] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.052273 9595 raft_consensus.cc:3060] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.052440 8553 raft_consensus.cc:2749] T ab1ec08da9f64109a3fe9400fe4e366c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251028 03:00:03.053228 10246 consensus_queue.cc:1048] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:03.053412 9595 raft_consensus.cc:2749] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.057704 8392 tablet_service.cc:1505] Processing CreateTablet for tablet e972e2d157134594938bfb0719c477ae (DEFAULT_TABLE table=default.Table_c8239b64a71949bb8553a15b778102b5 [id=071c10a40435470ab26285fe1b340e8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.058840 10246 raft_consensus.cc:493] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.058948 10246 raft_consensus.cc:515] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.059381 10246 leader_election.cc:290] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.060424 8275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6818c3f2dfa48448bb2d9b38ed0dc19" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:03.060740 8356 consensus_peers.cc:597] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20251028 03:00:03.060883 8356 leader_election.cc:343] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.060985 8550 tablet_replica.cc:333] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:03.061138 8550 raft_consensus.cc:2243] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.061266 8550 raft_consensus.cc:2272] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.061554 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6818c3f2dfa48448bb2d9b38ed0dc19" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:03.062053 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 524b582bd3bc46be92393ee933c860c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.062114 8550 ts_tablet_manager.cc:1916] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.064425 8550 ts_tablet_manager.cc:1929] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20251028 03:00:03.064417 8354 leader_election.cc:343] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.064491 8354 leader_election.cc:304] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.064740 10246 raft_consensus.cc:2749] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.065187 8550 log.cc:1199] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/b7d3062426434c339be4a745decae729
I20251028 03:00:03.065472 8550 ts_tablet_manager.cc:1950] T b7d3062426434c339be4a745decae729 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:03.065093 8544 tablet_bootstrap.cc:492] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.065680 8544 ts_tablet_manager.cc:1403] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.001s
I20251028 03:00:03.066195 8544 raft_consensus.cc:359] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.066298 8544 raft_consensus.cc:385] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.066326 8544 raft_consensus.cc:740] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.066444 8544 consensus_queue.cc:260] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.066735 8544 ts_tablet_manager.cc:1434] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:03.066897 8544 tablet_bootstrap.cc:492] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.068154 8544 tablet_bootstrap.cc:654] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.069440 8544 tablet_bootstrap.cc:492] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.069528 8544 ts_tablet_manager.cc:1403] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251028 03:00:03.070037 8544 raft_consensus.cc:359] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.070149 8544 raft_consensus.cc:385] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.070180 8544 raft_consensus.cc:740] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.070295 8544 consensus_queue.cc:260] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.070609 8544 ts_tablet_manager.cc:1434] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:03.070819 8544 tablet_bootstrap.cc:492] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.072467 8544 tablet_bootstrap.cc:654] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.075263 9595 raft_consensus.cc:493] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.075388 9595 raft_consensus.cc:515] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.075433 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e972e2d157134594938bfb0719c477ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.075814 9595 leader_election.cc:290] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:03.076293 8220 leader_election.cc:343] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: fd8c04efd8c2456b9c7643641c7fbc0d
W20251028 03:00:03.076386 8218 leader_election.cc:343] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: fd8c04efd8c2456b9c7643641c7fbc0d
I20251028 03:00:03.076462 8218 leader_election.cc:304] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.076610 9595 raft_consensus.cc:2749] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.078253 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49cc6983acdb430288ab0bb65145998e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.080838 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 362353a0ae094c53a81251bcc8ce09f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.083144 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3d365e4a834442aece57fe87b64bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.086373 8552 tablet_replica.cc:333] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.086429 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcc757c38ac6492e9697bcdd053364c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.086534 8552 raft_consensus.cc:2243] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.086644 8552 raft_consensus.cc:2272] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.086962 8546 tablet_bootstrap.cc:492] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.087337 8552 ts_tablet_manager.cc:1916] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.088023 8546 tablet_bootstrap.cc:654] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.088624 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d238a79585e481e97efee4ffd082421. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.089951 8552 ts_tablet_manager.cc:1929] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.090020 8552 log.cc:1199] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/fd8c04efd8c2456b9c7643641c7fbc0d
I20251028 03:00:03.090221 8552 ts_tablet_manager.cc:1950] T fd8c04efd8c2456b9c7643641c7fbc0d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.090781 8552 tablet_replica.cc:333] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.090909 8552 raft_consensus.cc:2243] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.090996 8552 raft_consensus.cc:2272] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.091708 8552 ts_tablet_manager.cc:1916] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.074791 10282 raft_consensus.cc:493] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9343ca8510de4eaa9945786e054a866d)
I20251028 03:00:03.093534 10282 raft_consensus.cc:515] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } }
I20251028 03:00:03.095085 10282 leader_election.cc:290] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aac264d6d2274d3283bd3251ab77e583 (127.7.154.189:45473), 9343ca8510de4eaa9945786e054a866d (127.7.154.188:44237)
I20251028 03:00:03.095228 8552 ts_tablet_manager.cc:1929] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.095470 8552 log.cc:1199] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9621e42e6a324ecda50fe83c0c53119c
I20251028 03:00:03.095816 8552 ts_tablet_manager.cc:1950] T 9621e42e6a324ecda50fe83c0c53119c P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.095844 9293 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac9bf5a198d042698c9f030a59228865" candidate_term: 3 candidate_status { last_received { term: 2 index: 1117 } } ignore_live_leader: false dest_uuid: "aac264d6d2274d3283bd3251ab77e583" is_pre_election: true
I20251028 03:00:03.096535 9595 raft_consensus.cc:493] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.096618 9595 raft_consensus.cc:515] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.096982 9595 leader_election.cc:290] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:03.097880 8552 tablet_replica.cc:333] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.095229 9808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9343ca8510de4eaa9945786e054a866d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9bf5a198d042698c9f030a59228865" member_type: VOTER last_known_addr { host: "127.7.154.190" port: 39091 } } peers { permanent_uuid: "aac264d6d2274d3283bd3251ab77e583" member_type: VOTER last_known_addr { host: "127.7.154.189" port: 45473 } } peers { permanent_uuid: "9343ca8510de4eaa9945786e054a866d" member_type: VOTER last_known_addr { host: "127.7.154.188" port: 44237 } } }
I20251028 03:00:03.098021 9808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d [sys.catalog]: This master's current role is: LEADER
I20251028 03:00:03.098090 8552 raft_consensus.cc:2243] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.098644 8552 raft_consensus.cc:2272] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.099315 8552 ts_tablet_manager.cc:1916] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:03.101171 8220 leader_election.cc:343] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 2ff112c66f1a4cf9b232e6bb665755f9
W20251028 03:00:03.101476 8218 leader_election.cc:343] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 2ff112c66f1a4cf9b232e6bb665755f9
I20251028 03:00:03.101548 8218 leader_election.cc:304] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.101733 8552 ts_tablet_manager.cc:1929] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.101806 8552 log.cc:1199] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/6bc54972260747f3af0c18d05c6a3442
I20251028 03:00:03.101979 9595 raft_consensus.cc:2749] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.102075 8552 ts_tablet_manager.cc:1950] T 6bc54972260747f3af0c18d05c6a3442 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.102851 8552 tablet_replica.cc:333] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.103132 8552 raft_consensus.cc:2243] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.103251 8552 raft_consensus.cc:2272] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.103695 8552 ts_tablet_manager.cc:1916] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.106000 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9277a4ffd0ca4bf7afd29e1c6ab33bc9 (table default.Table_eca57c87ce0240a0844cc210e4dbdd85 [id=289693853ee848e785d5460a21ae5063]) successfully deleted
W20251028 03:00:03.106271 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 already in progress: creating tablet
W20251028 03:00:03.106416 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 88cb16241e364f56b251d88079936c7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 88cb16241e364f56b251d88079936c7b already in progress: creating tablet
I20251028 03:00:03.106917 8552 ts_tablet_manager.cc:1929] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.107048 8552 log.cc:1199] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/2ff112c66f1a4cf9b232e6bb665755f9
I20251028 03:00:03.107450 8552 ts_tablet_manager.cc:1950] T 2ff112c66f1a4cf9b232e6bb665755f9 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.107647 10246 consensus_queue.cc:1048] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:03.108273 8552 tablet_replica.cc:333] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.108448 8552 raft_consensus.cc:2243] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:03.108649 8552 pending_rounds.cc:70] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 03:00:03.108750 8552 pending_rounds.cc:77] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597169347911680 op_type: NO_OP noop_request { }
I20251028 03:00:03.112411 8552 raft_consensus.cc:2889] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:03.112607 8552 raft_consensus.cc:2272] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.115522 8552 ts_tablet_manager.cc:1916] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.116518 10342 tablet.cc:1722] T ad926e9ee0044c62a7abbfca959a4f56 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:03.116595 9898 mvcc.cc:204] Tried to move back new op lower bound from 7215597170301902848 to 7215597168810049536. Current Snapshot: MvccSnapshot[applied={T|T < 7215597170301902848}]
I20251028 03:00:03.117460 10192 tablet.cc:1722] T ad926e9ee0044c62a7abbfca959a4f56 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:03.118166 10187 tablet.cc:1722] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 03:00:03.120719 8552 ts_tablet_manager.cc:1929] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:03.120888 8552 log.cc:1199] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/6599ec60c3004e349e7e576e4d13c4fe
I20251028 03:00:03.122673 8552 ts_tablet_manager.cc:1950] T 6599ec60c3004e349e7e576e4d13c4fe P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.123099 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 524b582bd3bc46be92393ee933c860c7 (DEFAULT_TABLE table=default.Table_555cd9726353408bbafada707d406c8d [id=d1d6a42ea4484ec1984883c8ede5e6cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.123446 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 524b582bd3bc46be92393ee933c860c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.125392 8552 tablet_replica.cc:333] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.125602 8552 raft_consensus.cc:2243] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:03.125804 8552 pending_rounds.cc:70] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 03:00:03.125849 8552 pending_rounds.cc:77] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597169547210752 op_type: NO_OP noop_request { }
I20251028 03:00:03.125928 8552 raft_consensus.cc:2889] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:03.125993 8552 raft_consensus.cc:2272] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.126487 8552 ts_tablet_manager.cc:1916] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.127763 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet ba64524142934af08658e434e7fb08ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.127956 8551 tablet_replica.cc:333] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.128114 8551 raft_consensus.cc:2243] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.128223 8551 raft_consensus.cc:2272] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.128705 8551 ts_tablet_manager.cc:1916] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.128705 8552 ts_tablet_manager.cc:1929] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:03.128784 8552 log.cc:1199] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/12d103e9b5024a87aa232c26e0c51ab9
I20251028 03:00:03.129097 8552 ts_tablet_manager.cc:1950] T 12d103e9b5024a87aa232c26e0c51ab9 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.131083 8551 ts_tablet_manager.cc:1929] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.131219 8551 log.cc:1199] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ba64524142934af08658e434e7fb08ed
I20251028 03:00:03.131505 8551 ts_tablet_manager.cc:1950] T ba64524142934af08658e434e7fb08ed P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 03:00:03.132761 7953 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.154.129:46529, user_credentials={real_user=slave}} blocked reactor thread for 154176us
I20251028 03:00:03.133353 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 1c41d49f9eba40a48bd6900671297db9 (table default.Table_46656faddd3141adbdd28b70154be5f4 [id=ab17c384ca9242f094e9373db2ef9c7a]) successfully deleted
I20251028 03:00:03.133599 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet b7d3062426434c339be4a745decae729 (table default.Table_4045ef6d27fd4f2fbfb8975411994305 [id=1dd28e8d2834401589250978261308c2]) successfully deleted
I20251028 03:00:03.133734 10245 consensus_queue.cc:1048] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:03.134310 8122 tablet_service.cc:1505] Processing CreateTablet for tablet bdd82979f1b9436fa59c2b92e0b06ec3 (DEFAULT_TABLE table=default.Table_be36f6bb04274166a890026a70c7dbf7 [id=48d1b74c935949bc818081a19c1589c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.134591 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd82979f1b9436fa59c2b92e0b06ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.138355 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet ad926e9ee0044c62a7abbfca959a4f56 (table default.Table_6ad31a79d9f043628bdac1dd2d4c263c [id=4570d7ddf33c447fb40d6411ea47e52a]). Expected version 1 got 0
I20251028 03:00:03.138527 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (table default.Table_93b5aac358764b869e8589ccc023ea0e [id=674f76f9afed4f3f9e2c165e2479d938]). Expected version 1 got 0
I20251028 03:00:03.138619 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet f422030a6a4042009efc7fcfb21a6bca (table default.Table_a01151cafd944efcbc12e5f654a7a05a [id=fdd6ecd598f240948882d710fd485da2]). Expected version 2 got 0
I20251028 03:00:03.138713 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 9a81841c22ae49c0a17931d90407accc (table default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]). Expected version 1 got 0
I20251028 03:00:03.139829 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_86f07803f4c84e848af88e14238e1407 [id=3815e9736bd34d668ec7f7e113e11e6f] alter complete (version 2)
I20251028 03:00:03.141319 8545 tablet_bootstrap.cc:492] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.141418 8545 ts_tablet_manager.cc:1403] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.096s user 0.000s sys 0.003s
I20251028 03:00:03.141965 8545 raft_consensus.cc:359] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.142071 8545 raft_consensus.cc:385] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.142104 8545 raft_consensus.cc:740] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.142217 8545 consensus_queue.cc:260] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.142503 8545 ts_tablet_manager.cc:1434] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:03.142668 8545 tablet_bootstrap.cc:492] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.143222 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c41d49f9eba40a48bd6900671297db9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.144968 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 5a8e8f1a333240c2b40a0e59c3a04172 (DEFAULT_TABLE table=default.Table_49f2ad3b1c934eaeab556927604ddf1b [id=e88ffc49924340aa92a3aedd631330bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.145243 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a8e8f1a333240c2b40a0e59c3a04172. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:03.145852 7953 catalog_manager.cc:4952] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 1c41d49f9eba40a48bd6900671297db9 because the tablet was not found. No further retry: Not found: Tablet not found: 1c41d49f9eba40a48bd6900671297db9
I20251028 03:00:03.147065 9595 consensus_queue.cc:1048] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:03.147239 8545 tablet_bootstrap.cc:654] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.147497 8121 tablet_service.cc:1505] Processing CreateTablet for tablet c0c270c782874a64a7d3fe3c97c90be7 (DEFAULT_TABLE table=default.Table_f96ae4167a29448e87878c857edcf307 [id=b5655d690fb14772af2601693960ddac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.147747 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c270c782874a64a7d3fe3c97c90be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.148954 8546 tablet_bootstrap.cc:492] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.149049 8546 ts_tablet_manager.cc:1403] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.062s user 0.002s sys 0.000s
I20251028 03:00:03.149372 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet fd8c04efd8c2456b9c7643641c7fbc0d (table default.Table_327424a7e56243ae97d58c0f6fd560fd [id=009632a3749c4bbe9cf4e856758b9a72]) successfully deleted
I20251028 03:00:03.149497 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9621e42e6a324ecda50fe83c0c53119c (table default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]) successfully deleted
I20251028 03:00:03.149530 8546 raft_consensus.cc:359] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.149608 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 6bc54972260747f3af0c18d05c6a3442 (table default.Table_4fe6d656356041dcaad42a9fd6f026d5 [id=f8375f32144c4541bea5dbcd51ab256e]) successfully deleted
I20251028 03:00:03.149657 8546 raft_consensus.cc:385] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.149688 8546 raft_consensus.cc:740] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.149740 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 2ff112c66f1a4cf9b232e6bb665755f9 (table default.Table_9f07958f85cf4923a67bf8ac1ad1c722 [id=26d281a132204bc288eda2ac08794956]) successfully deleted
I20251028 03:00:03.149842 8546 consensus_queue.cc:260] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.150125 8546 ts_tablet_manager.cc:1434] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.150290 8546 tablet_bootstrap.cc:492] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.151862 8546 tablet_bootstrap.cc:654] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.151862 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (table default.Table_93b5aac358764b869e8589ccc023ea0e [id=674f76f9afed4f3f9e2c165e2479d938]). Expected version 1 got 0
I20251028 03:00:03.151984 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet f422030a6a4042009efc7fcfb21a6bca (table default.Table_a01151cafd944efcbc12e5f654a7a05a [id=fdd6ecd598f240948882d710fd485da2]). Expected version 2 got 0
I20251028 03:00:03.152144 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 338cf9d02142436f97ccd44313cc8199 (table default.Table_77a37bbdfa4f41458e6b50fb213850bc [id=5f61b3c12ed647968fc2932aa24a13f3]). Expected version 2 got 0
I20251028 03:00:03.152225 8545 tablet_bootstrap.cc:492] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.152310 8545 ts_tablet_manager.cc:1403] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20251028 03:00:03.152338 7964 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 8665d2e933fa423d81a1b32f2ce8629b (table default.Table_139255bc4fb74fae8fb86563b54826bd [id=dc3721126b09466ab5efdb8bd4b46d89]). Expected version 1 got 0
I20251028 03:00:03.152940 8545 raft_consensus.cc:359] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.153064 8545 raft_consensus.cc:385] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.153095 8545 raft_consensus.cc:740] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.153216 8545 consensus_queue.cc:260] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.153525 8545 ts_tablet_manager.cc:1434] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:03.153725 7961 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_0d5bef57eadd45538258da2301f0cd31 [id=b0407c8f78a84245a954c67e888da3ec] alter complete (version 5)
I20251028 03:00:03.153841 9958 consensus_queue.cc:1048] T 4c0081e0ce4d4e499c37f217fe62c546 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:03.154938 8545 tablet_bootstrap.cc:492] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.156056 8545 tablet_bootstrap.cc:654] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.157837 8545 tablet_bootstrap.cc:492] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.157922 8545 ts_tablet_manager.cc:1403] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251028 03:00:03.158489 8545 raft_consensus.cc:359] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.158597 8545 raft_consensus.cc:385] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.158600 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet ad926e9ee0044c62a7abbfca959a4f56 (table default.Table_6ad31a79d9f043628bdac1dd2d4c263c [id=4570d7ddf33c447fb40d6411ea47e52a]). Expected version 1 got 0
I20251028 03:00:03.158634 8545 raft_consensus.cc:740] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.158707 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 338cf9d02142436f97ccd44313cc8199 (table default.Table_77a37bbdfa4f41458e6b50fb213850bc [id=5f61b3c12ed647968fc2932aa24a13f3]). Expected version 2 got 0
I20251028 03:00:03.158735 8545 consensus_queue.cc:260] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.158859 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (table default.Table_93b5aac358764b869e8589ccc023ea0e [id=674f76f9afed4f3f9e2c165e2479d938]). Expected version 1 got 0
I20251028 03:00:03.158926 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 8665d2e933fa423d81a1b32f2ce8629b (table default.Table_139255bc4fb74fae8fb86563b54826bd [id=dc3721126b09466ab5efdb8bd4b46d89]). Expected version 1 got 0
I20251028 03:00:03.159051 8545 ts_tablet_manager.cc:1434] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:03.159247 8545 tablet_bootstrap.cc:492] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.160441 8545 tablet_bootstrap.cc:654] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.160888 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 6599ec60c3004e349e7e576e4d13c4fe (table default.Table_4f4721dca71f432b82e320f8fa9b0cf5 [id=bc3f005bc9424552831e2ecca5d049a7]) successfully deleted
I20251028 03:00:03.161036 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 12d103e9b5024a87aa232c26e0c51ab9 (table default.Table_12b6b08740be470382334a74e5df73cb [id=1b3819ae7b6f4269a5be015632d5c786]) successfully deleted
I20251028 03:00:03.161178 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet fd8c04efd8c2456b9c7643641c7fbc0d (table default.Table_327424a7e56243ae97d58c0f6fd560fd [id=009632a3749c4bbe9cf4e856758b9a72]) successfully deleted
I20251028 03:00:03.161587 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 6599ec60c3004e349e7e576e4d13c4fe (table default.Table_4f4721dca71f432b82e320f8fa9b0cf5 [id=bc3f005bc9424552831e2ecca5d049a7]) successfully deleted
I20251028 03:00:03.161741 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 12d103e9b5024a87aa232c26e0c51ab9 (table default.Table_12b6b08740be470382334a74e5df73cb [id=1b3819ae7b6f4269a5be015632d5c786]) successfully deleted
W20251028 03:00:03.163327 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5fe5aa8ebc9c4ee6959d74c8c6050480 already in progress: creating tablet
W20251028 03:00:03.163460 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 88cb16241e364f56b251d88079936c7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 88cb16241e364f56b251d88079936c7b already in progress: creating tablet
I20251028 03:00:03.163549 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet f39ed19223fa4268a01aaf4c7762d86e (table default.Table_4debdeef26b14d9b93a878210eaba4c3 [id=5a391450a3a04f01bcd5c3c06d36cf66]) successfully deleted
I20251028 03:00:03.163630 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet b7d3062426434c339be4a745decae729 (table default.Table_4045ef6d27fd4f2fbfb8975411994305 [id=1dd28e8d2834401589250978261308c2]) successfully deleted
I20251028 03:00:03.163729 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8dab70ca2d9f4b5fba338cb337b0b396 (table default.Table_cff7897b7440432c96a505a493a0b149 [id=eab0e9b296d04340b7a2fd250f68d9f3]) successfully deleted
I20251028 03:00:03.163818 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 597d792ed13941a2a895b0dc305614f1 (table default.Table_2adb2ccc4e3e4c1396032eae34eeb18c [id=bf54b516275b46baa498e3b6a5150c97]) successfully deleted
I20251028 03:00:03.163916 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 1c41d49f9eba40a48bd6900671297db9 (table default.Table_46656faddd3141adbdd28b70154be5f4 [id=ab17c384ca9242f094e9373db2ef9c7a]) successfully deleted
I20251028 03:00:03.164122 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ba64524142934af08658e434e7fb08ed (table default.Table_8b7624342986474fbd0e31c86947f781 [id=530470c553154d76bf0df08ddfbf2fd0]) successfully deleted
I20251028 03:00:03.164755 8545 tablet_bootstrap.cc:492] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.164850 8545 ts_tablet_manager.cc:1403] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 03:00:03.165373 8545 raft_consensus.cc:359] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.165481 8545 raft_consensus.cc:385] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.165513 8545 raft_consensus.cc:740] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.165632 8545 consensus_queue.cc:260] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.165926 8545 ts_tablet_manager.cc:1434] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.166134 8545 tablet_bootstrap.cc:492] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.166247 8392 tablet_service.cc:1505] Processing CreateTablet for tablet fcc757c38ac6492e9697bcdd053364c6 (DEFAULT_TABLE table=default.Table_e9c0af79b3c44383883fa3c5ecfa256d [id=10af3345ea1f43668a7e444f6aa5ebaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.166505 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcc757c38ac6492e9697bcdd053364c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.164419 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_efe45b07a49349e7a7995ee1b0c4993e" } new_table_name: "default.Table_9278dbefdbdb4224acac84e97ee515b1" modify_external_catalogs: true
I20251028 03:00:03.167202 8545 tablet_bootstrap.cc:654] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.164487 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4d8bf6511df640c582ce134a8c146266" } modify_external_catalogs: true
I20251028 03:00:03.167589 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_4d8bf6511df640c582ce134a8c146266" } modify_external_catalogs: true
I20251028 03:00:03.167960 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_25c42d82559d47c6a78d15d5ffb4b4f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:03.164579 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d949a516b2954d27b669347dac4093bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.168327 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25c42d82559d47c6a78d15d5ffb4b4f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:03.168552 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d949a516b2954d27b669347dac4093bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.164654 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_adadaa41afba4f44ae002957844522c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.169353 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_adadaa41afba4f44ae002957844522c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.169565 8259 tablet_service.cc:1505] Processing CreateTablet for tablet bdd82979f1b9436fa59c2b92e0b06ec3 (DEFAULT_TABLE table=default.Table_be36f6bb04274166a890026a70c7dbf7 [id=48d1b74c935949bc818081a19c1589c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.169874 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd82979f1b9436fa59c2b92e0b06ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.165104 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_a27ed96bb5934fbd98f3015a38eff495"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:03.170065 8392 tablet_service.cc:1505] Processing CreateTablet for tablet bdd82979f1b9436fa59c2b92e0b06ec3 (DEFAULT_TABLE table=default.Table_be36f6bb04274166a890026a70c7dbf7 [id=48d1b74c935949bc818081a19c1589c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.170279 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd82979f1b9436fa59c2b92e0b06ec3. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:03.170301 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a27ed96bb5934fbd98f3015a38eff495 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.164705 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_d728be7dcc304b97afb19baa81c31e76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.172549 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d728be7dcc304b97afb19baa81c31e76 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.174113 8545 tablet_bootstrap.cc:492] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.174190 8545 ts_tablet_manager.cc:1403] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 03:00:03.174638 8545 raft_consensus.cc:359] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.174741 8545 raft_consensus.cc:385] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.174786 8545 raft_consensus.cc:740] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.174894 8545 consensus_queue.cc:260] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.175105 8545 ts_tablet_manager.cc:1434] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.175266 8545 tablet_bootstrap.cc:492] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.176316 8545 tablet_bootstrap.cc:654] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.176460 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 5a8e8f1a333240c2b40a0e59c3a04172 (DEFAULT_TABLE table=default.Table_49f2ad3b1c934eaeab556927604ddf1b [id=e88ffc49924340aa92a3aedd631330bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.176733 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a8e8f1a333240c2b40a0e59c3a04172. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.176964 7979 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac9bf5a198d042698c9f030a59228865" candidate_term: 3 candidate_status { last_received { term: 2 index: 1117 } } ignore_live_leader: false dest_uuid: "9343ca8510de4eaa9945786e054a866d" is_pre_election: true
I20251028 03:00:03.164300 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet f422030a6a4042009efc7fcfb21a6bca (table default.Table_a01151cafd944efcbc12e5f654a7a05a [id=fdd6ecd598f240948882d710fd485da2]). Expected version 2 got 0
I20251028 03:00:03.177923 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet e9a28ff8c0ee46cb89ba7e28de2cce45 (table default.Table_93b5aac358764b869e8589ccc023ea0e [id=674f76f9afed4f3f9e2c165e2479d938]). Expected version 1 got 0
I20251028 03:00:03.178027 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 8665d2e933fa423d81a1b32f2ce8629b (table default.Table_139255bc4fb74fae8fb86563b54826bd [id=dc3721126b09466ab5efdb8bd4b46d89]). Expected version 1 got 0
I20251028 03:00:03.178100 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet f62b981b9b1a44bb96bbc90f924c3b64 (table default.Table_3f85bca98f84492da9cc6b47c772334d [id=d4eaebd005bc42dd86d7125ef555546f]). Expected version 5 got 0
I20251028 03:00:03.178249 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 338cf9d02142436f97ccd44313cc8199 (table default.Table_77a37bbdfa4f41458e6b50fb213850bc [id=5f61b3c12ed647968fc2932aa24a13f3]). Expected version 2 got 0
I20251028 03:00:03.178351 7961 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet f6818c3f2dfa48448bb2d9b38ed0dc19 (table default.Table_73967d33588f4f02ab5e0010826ec5f6 [id=24424b561e55419cbc0d32089a83e21a]). Expected version 1 got 0
I20251028 03:00:03.178510 7964 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_6ad31a79d9f043628bdac1dd2d4c263c [id=4570d7ddf33c447fb40d6411ea47e52a] alter complete (version 1)
I20251028 03:00:03.178975 8252 tablet_service.cc:1505] Processing CreateTablet for tablet c0c270c782874a64a7d3fe3c97c90be7 (DEFAULT_TABLE table=default.Table_f96ae4167a29448e87878c857edcf307 [id=b5655d690fb14772af2601693960ddac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.179266 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c270c782874a64a7d3fe3c97c90be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.179519 10041 leader_election.cc:304] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac9bf5a198d042698c9f030a59228865; no voters: 9343ca8510de4eaa9945786e054a866d, aac264d6d2274d3283bd3251ab77e583
I20251028 03:00:03.179880 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_d246152b26344aed845d6366d95b40a0" } new_table_name: "default.Table_d8c0ae12755c45ba9e85292319669e32" modify_external_catalogs: true
I20251028 03:00:03.182225 8392 tablet_service.cc:1505] Processing CreateTablet for tablet 5a8e8f1a333240c2b40a0e59c3a04172 (DEFAULT_TABLE table=default.Table_49f2ad3b1c934eaeab556927604ddf1b [id=e88ffc49924340aa92a3aedd631330bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.182611 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a8e8f1a333240c2b40a0e59c3a04172. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.182804 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_42b51970061d47558809ca3152c09f84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.183624 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_42b51970061d47558809ca3152c09f84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.183647 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet f39ed19223fa4268a01aaf4c7762d86e with delete_type TABLET_DATA_DELETED (replaced by tablet b6a2791098934a9c8aa21d8db88f2c9e) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.183842 8550 tablet_replica.cc:333] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:03.183983 8550 raft_consensus.cc:2243] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.184099 8550 raft_consensus.cc:2272] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.184293 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8bca9ca8db1b491f9e6f7a564dc24ac1
I20251028 03:00:03.184590 8550 ts_tablet_manager.cc:1916] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.184672 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_16b568934eb1494e95ea301072e7d0d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.185026 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16b568934eb1494e95ea301072e7d0d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.185858 8545 tablet_bootstrap.cc:492] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.185946 8545 ts_tablet_manager.cc:1403] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 03:00:03.185947 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_46d5cdd6882c46da97c21a768a91d858" } new_table_name: "default.Table_19049cf1e5f2419fbff05d1df500a3ae" modify_external_catalogs: true
I20251028 03:00:03.186484 8545 raft_consensus.cc:359] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.186580 8545 raft_consensus.cc:385] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.186620 8545 raft_consensus.cc:740] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.186744 8550 ts_tablet_manager.cc:1929] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.186805 8550 log.cc:1199] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/f39ed19223fa4268a01aaf4c7762d86e
I20251028 03:00:03.186774 8545 consensus_queue.cc:260] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.186851 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_927bd3d6ebc54f3aaa7c47e636452f05" } modify_external_catalogs: true
I20251028 03:00:03.186929 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_927bd3d6ebc54f3aaa7c47e636452f05" } modify_external_catalogs: true
I20251028 03:00:03.187001 8550 ts_tablet_manager.cc:1950] T f39ed19223fa4268a01aaf4c7762d86e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:03.187139 8545 ts_tablet_manager.cc:1434] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.187390 8545 tablet_bootstrap.cc:492] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 03:00:03.187638 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e105f414dbb8487195ab201cfa914dec consensus state has no leader
I20251028 03:00:03.187836 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c295b736697341e795d7de52b865c4ab" } modify_external_catalogs: true
I20251028 03:00:03.187940 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c295b736697341e795d7de52b865c4ab" } modify_external_catalogs: true
I20251028 03:00:03.188906 8545 tablet_bootstrap.cc:654] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.189141 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet f39ed19223fa4268a01aaf4c7762d86e (table default.Table_4debdeef26b14d9b93a878210eaba4c3 [id=5a391450a3a04f01bcd5c3c06d36cf66]) successfully deleted
I20251028 03:00:03.189913 8544 tablet_bootstrap.cc:492] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.190017 8544 ts_tablet_manager.cc:1403] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.119s user 0.002s sys 0.000s
I20251028 03:00:03.190503 8544 raft_consensus.cc:359] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.190608 8544 raft_consensus.cc:385] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.190640 8544 raft_consensus.cc:740] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.190766 8544 consensus_queue.cc:260] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.191074 8544 ts_tablet_manager.cc:1434] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.191253 8544 tablet_bootstrap.cc:492] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.191594 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet ba64524142934af08658e434e7fb08ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.191747 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dab70ca2d9f4b5fba338cb337b0b396 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.192677 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 8bca9ca8db1b491f9e6f7a564dc24ac1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.192880 8550 tablet_replica.cc:333] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:03.193434 8550 raft_consensus.cc:2243] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.193482 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bb369e38928a4e20af6d144f13efd7ac" } modify_external_catalogs: true
I20251028 03:00:03.193584 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bb369e38928a4e20af6d144f13efd7ac" } modify_external_catalogs: true
I20251028 03:00:03.193539 8550 raft_consensus.cc:2272] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.195776 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_73757725ad844037851210da1a02399a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.196161 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73757725ad844037851210da1a02399a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.198769 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 338cf9d02142436f97ccd44313cc8199 (table default.Table_77a37bbdfa4f41458e6b50fb213850bc [id=5f61b3c12ed647968fc2932aa24a13f3]). Expected version 2 got 0
I20251028 03:00:03.200004 7964 master_service.cc:954] ReplaceTablet: received request to replace tablet ada2c7c883014aa597ad8dbe14bf51b3 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:03.200347 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 8bca9ca8db1b491f9e6f7a564dc24ac1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.200902 7962 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_713672ff0f77410cb29890c7e63471ad" } new_table_name: "default.Table_e220390a2b1c43a6bd357ebf915b7262" modify_external_catalogs: true
I20251028 03:00:03.201246 8551 tablet_replica.cc:333] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.201406 8551 raft_consensus.cc:2243] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.201517 8551 raft_consensus.cc:2272] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.201926 8551 ts_tablet_manager.cc:1916] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.202214 8550 ts_tablet_manager.cc:1916] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.203236 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet a4bcdc8a0c554b7cae81d7993e4f2861 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:03.205457 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b6dbdbfa2618496c8858dabf12f02366" } modify_external_catalogs: true
I20251028 03:00:03.205562 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b6dbdbfa2618496c8858dabf12f02366" } modify_external_catalogs: true
W20251028 03:00:03.206642 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4a57e122bbb94a9fac8c8254ab73a21f consensus state has no leader
I20251028 03:00:03.207907 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 26a60624333d43f1b701036b7e9dfdb0 (table default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3]). Expected version 1 got 0
W20251028 03:00:03.208566 7960 catalog_manager.cc:5841] Failed to send AlterTable RPC for tablet 26a60624333d43f1b701036b7e9dfdb0 (table default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 26a60624333d43f1b701036b7e9dfdb0 consensus state has no leader
I20251028 03:00:03.209992 8551 ts_tablet_manager.cc:1929] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:03.210062 8551 log.cc:1199] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8bca9ca8db1b491f9e6f7a564dc24ac1
I20251028 03:00:03.210301 8551 ts_tablet_manager.cc:1950] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.211123 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c8a55dcfb9e94100866bac84b2ce9ecd" } new_table_name: "default.Table_1aebb42ceeac4980ac2abbbc5f435a56" modify_external_catalogs: true
I20251028 03:00:03.211866 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9ea5e82c09354ea88fd67902625e1599
I20251028 03:00:03.212265 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ba24be41966c45ef969a0df13a3b2527"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.212580 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba24be41966c45ef969a0df13a3b2527 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.213600 7967 catalog_manager.cc:5931] Not sending DeleteTablet requests; no consensus state for tablet f9d1b4032a694a879b794216554f4413
I20251028 03:00:03.213775 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_2a5eacfbc4da4f6ebe5efca52a273f78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.214110 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a5eacfbc4da4f6ebe5efca52a273f78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.214967 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2
I20251028 03:00:03.215317 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet ee1d686a0e4c47f6b214224047a55747 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:03.215584 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.215615 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet 8bca9ca8db1b491f9e6f7a564dc24ac1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.216063 8552 tablet_replica.cc:333] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.216217 8552 raft_consensus.cc:2243] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:03.216403 8552 raft_consensus.cc:2272] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.216792 8552 ts_tablet_manager.cc:1916] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.217327 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 9ea5e82c09354ea88fd67902625e1599 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.218096 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ada2c7c883014aa597ad8dbe14bf51b3
I20251028 03:00:03.218364 7964 catalog_manager.cc:6719] ReplaceTablet: tablet ada2c7c883014aa597ad8dbe14bf51b3 deleted and replaced by tablet 2310a46fdcb2478faf8fdc5284aa5740
I20251028 03:00:03.218567 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet ada2c7c883014aa597ad8dbe14bf51b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2310a46fdcb2478faf8fdc5284aa5740) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.220304 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 566787825efe4f5aafd0d1f9442c8b26
I20251028 03:00:03.223322 8552 ts_tablet_manager.cc:1929] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:03.223416 8552 log.cc:1199] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/8bca9ca8db1b491f9e6f7a564dc24ac1
I20251028 03:00:03.223685 8550 ts_tablet_manager.cc:1929] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.224045 8550 log.cc:1199] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ba64524142934af08658e434e7fb08ed
I20251028 03:00:03.223922 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 566787825efe4f5aafd0d1f9442c8b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.224484 8550 ts_tablet_manager.cc:1950] T ba64524142934af08658e434e7fb08ed P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 03:00:03.224581 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet af93c341f684426fa8a847ee06d86d35 consensus state has no leader
W20251028 03:00:03.224917 7962 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cc67e4744b814540b79c07a20b6a3926 consensus state has no leader
I20251028 03:00:03.225128 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet ba64524142934af08658e434e7fb08ed (table default.Table_8b7624342986474fbd0e31c86947f781 [id=530470c553154d76bf0df08ddfbf2fd0]) successfully deleted
I20251028 03:00:03.225405 8550 tablet_replica.cc:333] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:03.225622 8550 raft_consensus.cc:2243] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.225814 8550 raft_consensus.cc:2272] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.226042 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet b7d3062426434c339be4a745decae729 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.226438 8550 ts_tablet_manager.cc:1916] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.229063 8550 ts_tablet_manager.cc:1929] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.229204 8550 log.cc:1199] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8dab70ca2d9f4b5fba338cb337b0b396
I20251028 03:00:03.229470 8550 ts_tablet_manager.cc:1950] T 8dab70ca2d9f4b5fba338cb337b0b396 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:03.231122 7961 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 2bf100407b034626b4a9f7e2340f9cf0 (table default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262]). Expected version 2 got 0
I20251028 03:00:03.231357 10242 raft_consensus.cc:493] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.231391 8552 ts_tablet_manager.cc:1950] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.231462 10242 raft_consensus.cc:515] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.231899 10242 leader_election.cc:290] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.232498 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a578008e735a4cc1a2f50dba09c16117" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.232663 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8dab70ca2d9f4b5fba338cb337b0b396 (table default.Table_cff7897b7440432c96a505a493a0b149 [id=eab0e9b296d04340b7a2fd250f68d9f3]) successfully deleted
W20251028 03:00:03.232870 8356 leader_election.cc:343] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.232947 8550 tablet_replica.cc:333] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:03.233235 8550 raft_consensus.cc:2243] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.233636 8550 raft_consensus.cc:2272] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.233649 7962 master_service.cc:954] ReplaceTablet: received request to replace tablet f03b90d615ab435f9a244504d8d01660 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:03.234134 8550 ts_tablet_manager.cc:1916] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.234169 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8bca9ca8db1b491f9e6f7a564dc24ac1 (table default.Table_4d8bf6511df640c582ce134a8c146266 [id=472a765337f346e9bbfb73717b8c9074]) successfully deleted
I20251028 03:00:03.234349 8385 tablet_service.cc:1505] Processing CreateTablet for tablet c0c270c782874a64a7d3fe3c97c90be7 (DEFAULT_TABLE table=default.Table_f96ae4167a29448e87878c857edcf307 [id=b5655d690fb14772af2601693960ddac]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.234642 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c270c782874a64a7d3fe3c97c90be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.235056 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a578008e735a4cc1a2f50dba09c16117" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:03.235874 8354 leader_election.cc:343] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.235952 8354 leader_election.cc:304] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.236147 10245 raft_consensus.cc:2749] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.236681 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 8483cbc3099644d2b6f7ff52869e21ea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.236852 8551 tablet_replica.cc:333] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.236982 8551 raft_consensus.cc:2243] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.237079 8551 raft_consensus.cc:2272] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.237260 8552 tablet_replica.cc:333] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.237404 8552 raft_consensus.cc:2243] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.237502 8552 raft_consensus.cc:2272] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.237684 8550 ts_tablet_manager.cc:1929] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:03.237823 8550 log.cc:1199] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8bca9ca8db1b491f9e6f7a564dc24ac1
I20251028 03:00:03.238152 8550 ts_tablet_manager.cc:1950] T 8bca9ca8db1b491f9e6f7a564dc24ac1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:03.238775 10187 tablet.cc:1722] T 2bf100407b034626b4a9f7e2340f9cf0 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:03.238821 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8bca9ca8db1b491f9e6f7a564dc24ac1 (table default.Table_4d8bf6511df640c582ce134a8c146266 [id=472a765337f346e9bbfb73717b8c9074]) successfully deleted
W20251028 03:00:03.239414 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 already in progress: creating tablet
I20251028 03:00:03.239625 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet a4bcdc8a0c554b7cae81d7993e4f2861 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:03.239925 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9ea5e82c09354ea88fd67902625e1599 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ea5e82c09354ea88fd67902625e1599 already in progress: creating tablet
W20251028 03:00:03.240310 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet ada2c7c883014aa597ad8dbe14bf51b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ada2c7c883014aa597ad8dbe14bf51b3 already in progress: creating tablet
W20251028 03:00:03.240662 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 566787825efe4f5aafd0d1f9442c8b26 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 566787825efe4f5aafd0d1f9442c8b26 already in progress: creating tablet
I20251028 03:00:03.240893 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet e369dbabc2c04ad5ab17bbd4225127c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.241459 8552 ts_tablet_manager.cc:1916] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.241978 10192 tablet.cc:1722] T 2bf100407b034626b4a9f7e2340f9cf0 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:03.242498 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.241994 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 9ea5e82c09354ea88fd67902625e1599 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.242960 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.244138 8544 tablet_bootstrap.cc:654] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.244722 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet 9ea5e82c09354ea88fd67902625e1599 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.245237 8552 ts_tablet_manager.cc:1929] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.246650 8121 tablet_service.cc:1505] Processing CreateTablet for tablet f9425f2912d34c9da2631056f7c43cc2 (DEFAULT_TABLE table=default.Table_dbaf1e8395cd43a688b9952597cb6e8c [id=5c59b015d8f14aad8c6d47ec913553b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.246927 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9425f2912d34c9da2631056f7c43cc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.247797 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet ada2c7c883014aa597ad8dbe14bf51b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2310a46fdcb2478faf8fdc5284aa5740) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.250854 8544 tablet_bootstrap.cc:492] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.250954 8544 ts_tablet_manager.cc:1403] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.060s user 0.003s sys 0.000s
I20251028 03:00:03.250958 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 566787825efe4f5aafd0d1f9442c8b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.251489 8544 raft_consensus.cc:359] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.251600 8544 raft_consensus.cc:385] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.251636 8544 raft_consensus.cc:740] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.251747 8544 consensus_queue.cc:260] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.251973 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ad4ae17665483584ce2d638d1639fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.252053 8544 ts_tablet_manager.cc:1434] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.252683 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 427ae7e31e81447ba6f8c8b8ccf4761c from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:03.252748 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 8bca9ca8db1b491f9e6f7a564dc24ac1 (table default.Table_4d8bf6511df640c582ce134a8c146266 [id=472a765337f346e9bbfb73717b8c9074]) successfully deleted
I20251028 03:00:03.254510 8385 tablet_service.cc:1505] Processing CreateTablet for tablet f9425f2912d34c9da2631056f7c43cc2 (DEFAULT_TABLE table=default.Table_dbaf1e8395cd43a688b9952597cb6e8c [id=5c59b015d8f14aad8c6d47ec913553b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.254813 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9425f2912d34c9da2631056f7c43cc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.256861 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 597d792ed13941a2a895b0dc305614f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.256948 8122 tablet_service.cc:1505] Processing CreateTablet for tablet f6400fa6c8f94e7fa50b221a2aa323f0 (DEFAULT_TABLE table=default.Table_6f8c1a5b100b4e0b8db85770d0a3108d [id=935bf884379e4c26a11a52e032eeab43]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.257330 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6400fa6c8f94e7fa50b221a2aa323f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.257522 8544 tablet_bootstrap.cc:492] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 03:00:03.258167 8083 consensus_peers.cc:597] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:03.258394 8544 tablet_bootstrap.cc:654] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.258504 10342 tablet.cc:1722] T 2bf100407b034626b4a9f7e2340f9cf0 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:03.258744 8392 tablet_service.cc:1505] Processing CreateTablet for tablet c2a10e18d9c4467b91cb380cc9d0cb01 (DEFAULT_TABLE table=default.Table_56137eddf5dd461c8ef068d0dabb832c [id=82e9e97ebff240188fe8acb10ef3ff8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.258988 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2a10e18d9c4467b91cb380cc9d0cb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.259756 8551 ts_tablet_manager.cc:1916] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.261049 8387 tablet_service.cc:1505] Processing CreateTablet for tablet f6400fa6c8f94e7fa50b221a2aa323f0 (DEFAULT_TABLE table=default.Table_6f8c1a5b100b4e0b8db85770d0a3108d [id=935bf884379e4c26a11a52e032eeab43]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.262670 8121 tablet_service.cc:1505] Processing CreateTablet for tablet c2a10e18d9c4467b91cb380cc9d0cb01 (DEFAULT_TABLE table=default.Table_56137eddf5dd461c8ef068d0dabb832c [id=82e9e97ebff240188fe8acb10ef3ff8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.262849 8550 tablet_replica.cc:333] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:03.262991 8550 raft_consensus.cc:2243] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.263104 8550 raft_consensus.cc:2272] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.263388 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6400fa6c8f94e7fa50b221a2aa323f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.263473 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2a10e18d9c4467b91cb380cc9d0cb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.264325 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_24ec851d131d4496955e5cb7f05d562e [id=51fb1b3116c14113b2e14ced3ac17262] alter complete (version 2)
I20251028 03:00:03.265167 8550 ts_tablet_manager.cc:1916] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.267163 8550 ts_tablet_manager.cc:1929] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.267643 8550 log.cc:1199] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/597d792ed13941a2a895b0dc305614f1
I20251028 03:00:03.267938 8550 ts_tablet_manager.cc:1950] T 597d792ed13941a2a895b0dc305614f1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:03.268599 8551 ts_tablet_manager.cc:1929] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.268658 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 597d792ed13941a2a895b0dc305614f1 (table default.Table_2adb2ccc4e3e4c1396032eae34eeb18c [id=bf54b516275b46baa498e3b6a5150c97]) successfully deleted
I20251028 03:00:03.268764 8551 log.cc:1199] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8483cbc3099644d2b6f7ff52869e21ea
I20251028 03:00:03.269014 8551 ts_tablet_manager.cc:1950] T 8483cbc3099644d2b6f7ff52869e21ea P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.269744 8551 tablet_replica.cc:333] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.269955 8551 raft_consensus.cc:2243] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.270272 8551 raft_consensus.cc:2272] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.271008 10192 tablet.cc:1722] T 73a45776f8964e68b79bc973097223a3 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 03:00:03.271402 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 427ae7e31e81447ba6f8c8b8ccf4761c
I20251028 03:00:03.271847 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 427ae7e31e81447ba6f8c8b8ccf4761c with delete_type TABLET_DATA_DELETED (replaced by tablet df45f016317b43d5a3219fd2e481756a) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.272012 10342 tablet.cc:1722] T 73a45776f8964e68b79bc973097223a3 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 03:00:03.272361 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 427ae7e31e81447ba6f8c8b8ccf4761c deleted and replaced by tablet df45f016317b43d5a3219fd2e481756a
I20251028 03:00:03.272536 8551 ts_tablet_manager.cc:1916] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.274937 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 427ae7e31e81447ba6f8c8b8ccf4761c with delete_type TABLET_DATA_DELETED (replaced by tablet df45f016317b43d5a3219fd2e481756a) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.275583 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8483cbc3099644d2b6f7ff52869e21ea (table default.Table_8d295e2ce9374929ab925a8d8329920d [id=de05ccfb39064421868853d6d9866bda]) successfully deleted
I20251028 03:00:03.278671 8551 ts_tablet_manager.cc:1929] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.278765 8551 log.cc:1199] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/a4bcdc8a0c554b7cae81d7993e4f2861
I20251028 03:00:03.278970 8551 ts_tablet_manager.cc:1950] T a4bcdc8a0c554b7cae81d7993e4f2861 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.247756 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet ada2c7c883014aa597ad8dbe14bf51b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2310a46fdcb2478faf8fdc5284aa5740) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.279767 8552 log.cc:1199] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/b7d3062426434c339be4a745decae729
I20251028 03:00:03.279801 8546 tablet_bootstrap.cc:492] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.251830 8258 tablet_service.cc:1505] Processing CreateTablet for tablet f9425f2912d34c9da2631056f7c43cc2 (DEFAULT_TABLE table=default.Table_dbaf1e8395cd43a688b9952597cb6e8c [id=5c59b015d8f14aad8c6d47ec913553b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.258738 8249 tablet_service.cc:1505] Processing CreateTablet for tablet c2a10e18d9c4467b91cb380cc9d0cb01 (DEFAULT_TABLE table=default.Table_56137eddf5dd461c8ef068d0dabb832c [id=82e9e97ebff240188fe8acb10ef3ff8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.279918 8546 ts_tablet_manager.cc:1403] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.130s user 0.003s sys 0.000s
I20251028 03:00:03.261004 8251 tablet_service.cc:1505] Processing CreateTablet for tablet f6400fa6c8f94e7fa50b221a2aa323f0 (DEFAULT_TABLE table=default.Table_6f8c1a5b100b4e0b8db85770d0a3108d [id=935bf884379e4c26a11a52e032eeab43]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.272486 10187 tablet.cc:1722] T 73a45776f8964e68b79bc973097223a3 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20251028 03:00:03.280081 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9425f2912d34c9da2631056f7c43cc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.274726 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 427ae7e31e81447ba6f8c8b8ccf4761c with delete_type TABLET_DATA_DELETED (replaced by tablet df45f016317b43d5a3219fd2e481756a) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.250793 8259 tablet_service.cc:1552] Processing DeleteTablet for tablet 566787825efe4f5aafd0d1f9442c8b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.280434 8546 raft_consensus.cc:359] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.280553 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6400fa6c8f94e7fa50b221a2aa323f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.280542 8546 raft_consensus.cc:385] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.281241 8546 raft_consensus.cc:740] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.281397 8546 consensus_queue.cc:260] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.281721 8546 ts_tablet_manager.cc:1434] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:03.281925 8546 tablet_bootstrap.cc:492] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.282737 8552 ts_tablet_manager.cc:1950] T b7d3062426434c339be4a745decae729 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.282968 8546 tablet_bootstrap.cc:654] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:03.283902 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 427ae7e31e81447ba6f8c8b8ccf4761c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 427ae7e31e81447ba6f8c8b8ccf4761c already in progress: creating tablet
I20251028 03:00:03.288692 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2a10e18d9c4467b91cb380cc9d0cb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.288784 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet 1c41d49f9eba40a48bd6900671297db9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:56 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.289049 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet b7d3062426434c339be4a745decae729 (table default.Table_4045ef6d27fd4f2fbfb8975411994305 [id=1dd28e8d2834401589250978261308c2]) successfully deleted
W20251028 03:00:03.289463 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 already in progress: creating tablet
W20251028 03:00:03.292347 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 9ea5e82c09354ea88fd67902625e1599 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ea5e82c09354ea88fd67902625e1599 already in progress: creating tablet
W20251028 03:00:03.293185 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet ada2c7c883014aa597ad8dbe14bf51b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ada2c7c883014aa597ad8dbe14bf51b3 already in progress: creating tablet
W20251028 03:00:03.293358 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 427ae7e31e81447ba6f8c8b8ccf4761c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 427ae7e31e81447ba6f8c8b8ccf4761c already in progress: creating tablet
I20251028 03:00:03.293632 8552 tablet_replica.cc:333] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.293967 8551 tablet_replica.cc:333] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.294159 8552 raft_consensus.cc:2243] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.294514 8551 raft_consensus.cc:2243] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.294696 8552 raft_consensus.cc:2272] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.296379 7962 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_d8c0ae12755c45ba9e85292319669e32 [id=a9b54811e6ba4461a372d76424f07b4f] alter complete (version 3)
W20251028 03:00:03.294348 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 566787825efe4f5aafd0d1f9442c8b26 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 566787825efe4f5aafd0d1f9442c8b26 already in progress: creating tablet
I20251028 03:00:03.296921 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet a4bcdc8a0c554b7cae81d7993e4f2861 (table default.Table_806d85836a4c44a986095f5108a93d7f [id=c098c2727dd147e69471353bd9a741fc]) successfully deleted
I20251028 03:00:03.296125 8545 tablet_bootstrap.cc:492] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.297101 8545 ts_tablet_manager.cc:1403] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.110s user 0.000s sys 0.002s
I20251028 03:00:03.297576 8545 raft_consensus.cc:359] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.297688 8545 raft_consensus.cc:385] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.297729 8545 raft_consensus.cc:740] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.297837 8545 consensus_queue.cc:260] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.297978 8544 tablet_bootstrap.cc:492] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.298082 8544 ts_tablet_manager.cc:1403] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.041s user 0.001s sys 0.000s
I20251028 03:00:03.298095 8545 ts_tablet_manager.cc:1434] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:03.298274 8545 tablet_bootstrap.cc:492] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.298568 8544 raft_consensus.cc:359] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.298668 8544 raft_consensus.cc:385] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.298694 8544 raft_consensus.cc:740] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.300344 8544 consensus_queue.cc:260] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.299276 8545 tablet_bootstrap.cc:654] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.301556 8551 raft_consensus.cc:2272] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.297380 8546 tablet_bootstrap.cc:492] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.301765 8546 ts_tablet_manager.cc:1403] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.002s
I20251028 03:00:03.302296 8546 raft_consensus.cc:359] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.302416 8546 raft_consensus.cc:385] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.302451 8546 raft_consensus.cc:740] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.302572 8546 consensus_queue.cc:260] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.302856 8546 ts_tablet_manager.cc:1434] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.303052 8546 tablet_bootstrap.cc:492] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.303808 7961 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet f6818c3f2dfa48448bb2d9b38ed0dc19 (table default.Table_73967d33588f4f02ab5e0010826ec5f6 [id=24424b561e55419cbc0d32089a83e21a]). Expected version 1 got 0
I20251028 03:00:03.304188 8546 tablet_bootstrap.cc:654] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.304528 8544 ts_tablet_manager.cc:1434] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251028 03:00:03.304734 8544 tablet_bootstrap.cc:492] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.307025 7967 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet f6818c3f2dfa48448bb2d9b38ed0dc19 (table default.Table_73967d33588f4f02ab5e0010826ec5f6 [id=24424b561e55419cbc0d32089a83e21a]). Expected version 1 got 0
I20251028 03:00:03.307826 8551 ts_tablet_manager.cc:1916] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.309759 8249 tablet_service.cc:1552] Processing DeleteTablet for tablet ba64524142934af08658e434e7fb08ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.310288 8552 ts_tablet_manager.cc:1916] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.313577 8544 tablet_bootstrap.cc:654] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.314826 8551 ts_tablet_manager.cc:1929] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.314905 8551 log.cc:1199] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/e369dbabc2c04ad5ab17bbd4225127c1
I20251028 03:00:03.315102 8551 ts_tablet_manager.cc:1950] T e369dbabc2c04ad5ab17bbd4225127c1 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.319339 8546 tablet_bootstrap.cc:492] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.319453 8546 ts_tablet_manager.cc:1403] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 03:00:03.320137 8546 raft_consensus.cc:359] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.320256 8546 raft_consensus.cc:385] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.320292 8546 raft_consensus.cc:740] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.320406 8546 consensus_queue.cc:260] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.320685 8546 ts_tablet_manager.cc:1434] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.320871 8546 tablet_bootstrap.cc:492] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.323776 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet e369dbabc2c04ad5ab17bbd4225127c1 (table default.Table_621dda8a15654228acd735a8dca754ef [id=36f3b29b4d154e6caf4c3fb52bc82466]) successfully deleted
W20251028 03:00:03.324275 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4043fc8c6c4547a59d70e2d6dbf1a3f2 already in progress: creating tablet
W20251028 03:00:03.324625 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 9ea5e82c09354ea88fd67902625e1599 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ea5e82c09354ea88fd67902625e1599 already in progress: creating tablet
W20251028 03:00:03.325052 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet ada2c7c883014aa597ad8dbe14bf51b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ada2c7c883014aa597ad8dbe14bf51b3 already in progress: creating tablet
W20251028 03:00:03.325402 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 566787825efe4f5aafd0d1f9442c8b26 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 566787825efe4f5aafd0d1f9442c8b26 already in progress: creating tablet
I20251028 03:00:03.325563 8546 tablet_bootstrap.cc:654] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.325616 8551 tablet_replica.cc:333] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.325850 8551 raft_consensus.cc:2243] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.326031 8551 raft_consensus.cc:2272] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.326586 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet e3b33c4d189247a89878b6a8e1fcbac0 (table default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]). Expected version 1 got 0
I20251028 03:00:03.326792 8545 tablet_bootstrap.cc:492] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.326871 8545 ts_tablet_manager.cc:1403] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20251028 03:00:03.327339 8545 raft_consensus.cc:359] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.327448 8545 raft_consensus.cc:385] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.327486 8545 raft_consensus.cc:740] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.327621 8545 consensus_queue.cc:260] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.327913 8545 ts_tablet_manager.cc:1434] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.328089 8545 tablet_bootstrap.cc:492] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.329584 8545 tablet_bootstrap.cc:654] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.330309 10282 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ac9bf5a198d042698c9f030a59228865 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20251028 03:00:03.330691 8552 ts_tablet_manager.cc:1929] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.330770 8552 log.cc:1199] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/1c41d49f9eba40a48bd6900671297db9
I20251028 03:00:03.330982 8552 ts_tablet_manager.cc:1950] T 1c41d49f9eba40a48bd6900671297db9 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.332667 8544 tablet_bootstrap.cc:492] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.332760 8544 ts_tablet_manager.cc:1403] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20251028 03:00:03.333304 8544 raft_consensus.cc:359] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.333415 8544 raft_consensus.cc:385] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.333447 8544 raft_consensus.cc:740] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.333567 8544 consensus_queue.cc:260] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.333878 8544 ts_tablet_manager.cc:1434] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.334076 8544 tablet_bootstrap.cc:492] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.334251 8552 tablet_replica.cc:333] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.334403 8552 raft_consensus.cc:2243] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.334713 8552 raft_consensus.cc:2272] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.336529 8544 tablet_bootstrap.cc:654] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.338632 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 1c41d49f9eba40a48bd6900671297db9 (table default.Table_46656faddd3141adbdd28b70154be5f4 [id=ab17c384ca9242f094e9373db2ef9c7a]) successfully deleted
I20251028 03:00:03.339521 8551 ts_tablet_manager.cc:1916] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.341946 8551 ts_tablet_manager.cc:1929] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.342294 10242 raft_consensus.cc:493] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.343616 10242 raft_consensus.cc:515] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.344028 10242 leader_election.cc:290] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.344614 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a60624333d43f1b701036b7e9dfdb0" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.344898 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a60624333d43f1b701036b7e9dfdb0" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:03.345059 8356 leader_election.cc:343] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:03.345460 8354 leader_election.cc:343] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.345520 8354 leader_election.cc:304] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.345777 10242 raft_consensus.cc:2749] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.346074 8551 log.cc:1199] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/e9ad4ae17665483584ce2d638d1639fb
I20251028 03:00:03.346220 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet a38ef10c145a4d00a26265a9dee5c744 (table default.Table_8d72f9b8a87847d0a40f66c2bcdba651 [id=ba8b6a629ebc489abe7043890455765a]). Expected version 1 got 0
I20251028 03:00:03.346566 8551 ts_tablet_manager.cc:1950] T e9ad4ae17665483584ce2d638d1639fb P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.348039 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet e9ad4ae17665483584ce2d638d1639fb (table default.Table_929b382a93794515a84b4f6356fb86b2 [id=5f35ae4ce3914890a3ce64198fbdb39e]) successfully deleted
I20251028 03:00:03.348450 8552 ts_tablet_manager.cc:1916] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:03.349923 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 427ae7e31e81447ba6f8c8b8ccf4761c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 427ae7e31e81447ba6f8c8b8ccf4761c already in progress: creating tablet
I20251028 03:00:03.350639 8546 tablet_bootstrap.cc:492] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.350744 8546 ts_tablet_manager.cc:1403] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20251028 03:00:03.351226 8546 raft_consensus.cc:359] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.351346 8546 raft_consensus.cc:385] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.351374 8546 raft_consensus.cc:740] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.351487 8546 consensus_queue.cc:260] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.351771 8546 ts_tablet_manager.cc:1434] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.352173 8546 tablet_bootstrap.cc:492] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.352375 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet a38ef10c145a4d00a26265a9dee5c744 (table default.Table_8d72f9b8a87847d0a40f66c2bcdba651 [id=ba8b6a629ebc489abe7043890455765a]). Expected version 1 got 0
I20251028 03:00:03.353211 8546 tablet_bootstrap.cc:654] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.353952 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet f62b981b9b1a44bb96bbc90f924c3b64 (table default.Table_3f85bca98f84492da9cc6b47c772334d [id=d4eaebd005bc42dd86d7125ef555546f]). Expected version 5 got 0
I20251028 03:00:03.354367 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet 28b81f0000c4407096feff773c29579d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.354589 8551 tablet_replica.cc:333] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.355103 8545 tablet_bootstrap.cc:492] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.355234 8545 ts_tablet_manager.cc:1403] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.000s
I20251028 03:00:03.355788 8545 raft_consensus.cc:359] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.355890 8545 raft_consensus.cc:385] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.355919 8545 raft_consensus.cc:740] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.355944 8121 tablet_service.cc:1505] Processing CreateTablet for tablet e1ded23c515e4e27bfa803d6e1b3b377 (DEFAULT_TABLE table=default.Table_ba24be41966c45ef969a0df13a3b2527 [id=2cb835da04034db18ba3a9b814f8a5f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.356040 8545 consensus_queue.cc:260] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.356244 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ded23c515e4e27bfa803d6e1b3b377. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.356314 8545 ts_tablet_manager.cc:1434] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.356511 8545 tablet_bootstrap.cc:492] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.357648 8545 tablet_bootstrap.cc:654] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.357618 8111 tablet_service.cc:1505] Processing CreateTablet for tablet fff6e3fe68aa43ed9fb287815e127a2a (DEFAULT_TABLE table=default.Table_d728be7dcc304b97afb19baa81c31e76 [id=2dc54ddbe04b4d8bbc257835f98d2262]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.357846 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fff6e3fe68aa43ed9fb287815e127a2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.358608 8122 tablet_service.cc:1505] Processing CreateTablet for tablet f5d83c6850714eceb0ec56166a25b463 (DEFAULT_TABLE table=default.Table_16b568934eb1494e95ea301072e7d0d4 [id=27719680965c44d187542b936bfd2314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.358837 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5d83c6850714eceb0ec56166a25b463. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.360242 8385 tablet_service.cc:1505] Processing CreateTablet for tablet e1ded23c515e4e27bfa803d6e1b3b377 (DEFAULT_TABLE table=default.Table_ba24be41966c45ef969a0df13a3b2527 [id=2cb835da04034db18ba3a9b814f8a5f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.360519 8387 tablet_service.cc:1505] Processing CreateTablet for tablet f5d83c6850714eceb0ec56166a25b463 (DEFAULT_TABLE table=default.Table_16b568934eb1494e95ea301072e7d0d4 [id=27719680965c44d187542b936bfd2314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.360885 8249 tablet_service.cc:1552] Processing DeleteTablet for tablet 8dab70ca2d9f4b5fba338cb337b0b396 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.361146 8258 tablet_service.cc:1505] Processing CreateTablet for tablet f5d83c6850714eceb0ec56166a25b463 (DEFAULT_TABLE table=default.Table_16b568934eb1494e95ea301072e7d0d4 [id=27719680965c44d187542b936bfd2314]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.361147 8251 tablet_service.cc:1505] Processing CreateTablet for tablet e1ded23c515e4e27bfa803d6e1b3b377 (DEFAULT_TABLE table=default.Table_ba24be41966c45ef969a0df13a3b2527 [id=2cb835da04034db18ba3a9b814f8a5f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.361416 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5d83c6850714eceb0ec56166a25b463. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.361514 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ded23c515e4e27bfa803d6e1b3b377. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.363884 8546 tablet_bootstrap.cc:492] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.363976 8546 ts_tablet_manager.cc:1403] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 03:00:03.364446 8546 raft_consensus.cc:359] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.364552 8546 raft_consensus.cc:385] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.364583 8546 raft_consensus.cc:740] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.364703 8546 consensus_queue.cc:260] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.364795 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 32efb7b57678451f97abea5ea61da87a (DEFAULT_TABLE table=default.Table_25c42d82559d47c6a78d15d5ffb4b4f7 [id=2a1f0d60742448e59c61a93d2a508ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.365058 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32efb7b57678451f97abea5ea61da87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.366180 8392 tablet_service.cc:1505] Processing CreateTablet for tablet fff6e3fe68aa43ed9fb287815e127a2a (DEFAULT_TABLE table=default.Table_d728be7dcc304b97afb19baa81c31e76 [id=2dc54ddbe04b4d8bbc257835f98d2262]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.366470 8386 tablet_service.cc:1505] Processing CreateTablet for tablet eb9beea75b364425b897ce075388fe64 (DEFAULT_TABLE table=default.Table_adadaa41afba4f44ae002957844522c2 [id=0ac2a91a267a4b96866f3d14a067aed2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.367486 8122 tablet_service.cc:1505] Processing CreateTablet for tablet eb9beea75b364425b897ce075388fe64 (DEFAULT_TABLE table=default.Table_adadaa41afba4f44ae002957844522c2 [id=0ac2a91a267a4b96866f3d14a067aed2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.367488 8251 tablet_service.cc:1505] Processing CreateTablet for tablet fff6e3fe68aa43ed9fb287815e127a2a (DEFAULT_TABLE table=default.Table_d728be7dcc304b97afb19baa81c31e76 [id=2dc54ddbe04b4d8bbc257835f98d2262]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.367722 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fff6e3fe68aa43ed9fb287815e127a2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.367722 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb9beea75b364425b897ce075388fe64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.368017 8249 tablet_service.cc:1505] Processing CreateTablet for tablet eb9beea75b364425b897ce075388fe64 (DEFAULT_TABLE table=default.Table_adadaa41afba4f44ae002957844522c2 [id=0ac2a91a267a4b96866f3d14a067aed2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.368252 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb9beea75b364425b897ce075388fe64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.369671 8551 raft_consensus.cc:2243] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.370556 8546 ts_tablet_manager.cc:1434] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20251028 03:00:03.370687 8551 raft_consensus.cc:2272] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.370770 8546 tablet_bootstrap.cc:492] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.372551 8552 ts_tablet_manager.cc:1929] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.373139 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ded23c515e4e27bfa803d6e1b3b377. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.374359 8552 log.cc:1199] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/ba64524142934af08658e434e7fb08ed
I20251028 03:00:03.372813 8546 tablet_bootstrap.cc:654] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.375512 8552 ts_tablet_manager.cc:1950] T ba64524142934af08658e434e7fb08ed P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.375783 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5d83c6850714eceb0ec56166a25b463. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.376107 8552 tablet_replica.cc:333] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.376252 8552 raft_consensus.cc:2243] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.376354 8552 raft_consensus.cc:2272] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.376962 8544 tablet_bootstrap.cc:492] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.377041 8544 ts_tablet_manager.cc:1403] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.000s
I20251028 03:00:03.377398 8544 raft_consensus.cc:359] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.377482 8544 raft_consensus.cc:385] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.377502 8544 raft_consensus.cc:740] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.377629 8544 consensus_queue.cc:260] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.377907 8544 ts_tablet_manager.cc:1434] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.378981 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 58268ac590f3437a942dcafd927a5496 (table default.Table_0b7b98e5884a48de833ab1fbbad97f08 [id=fdb289a561dd4c6497173ef43e2e9ccd]). Expected version 1 got 0
I20251028 03:00:03.379652 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fff6e3fe68aa43ed9fb287815e127a2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.379668 8544 tablet_bootstrap.cc:492] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.376564 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 23c034566e22488b898c495c71e2eeaa (DEFAULT_TABLE table=default.Table_5b46e89d6c094f008e7da4d4ae8b6c7b [id=b1bbdd84dec6405e9dabc4e722e22734]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.380438 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23c034566e22488b898c495c71e2eeaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.381913 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb9beea75b364425b897ce075388fe64. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.383240 8544 tablet_bootstrap.cc:654] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.387024 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 8483cbc3099644d2b6f7ff52869e21ea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.388375 8550 tablet_replica.cc:333] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:03.388556 8550 raft_consensus.cc:2243] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.388674 8550 raft_consensus.cc:2272] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.391391 8111 tablet_service.cc:1505] Processing CreateTablet for tablet c90d9bc6893148b6a314f9e9036c6359 (DEFAULT_TABLE table=default.Table_5584a2f654f14984b2d85d2c44dbc692 [id=8ad209ab13e542a3a30e1790d770795c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.391677 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c90d9bc6893148b6a314f9e9036c6359. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.394428 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 32efb7b57678451f97abea5ea61da87a (DEFAULT_TABLE table=default.Table_25c42d82559d47c6a78d15d5ffb4b4f7 [id=2a1f0d60742448e59c61a93d2a508ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.394740 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32efb7b57678451f97abea5ea61da87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.396675 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 23c034566e22488b898c495c71e2eeaa (DEFAULT_TABLE table=default.Table_5b46e89d6c094f008e7da4d4ae8b6c7b [id=b1bbdd84dec6405e9dabc4e722e22734]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.396952 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23c034566e22488b898c495c71e2eeaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.400277 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 32efb7b57678451f97abea5ea61da87a (DEFAULT_TABLE table=default.Table_25c42d82559d47c6a78d15d5ffb4b4f7 [id=2a1f0d60742448e59c61a93d2a508ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.400565 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32efb7b57678451f97abea5ea61da87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.400897 7960 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 58268ac590f3437a942dcafd927a5496 (table default.Table_0b7b98e5884a48de833ab1fbbad97f08 [id=fdb289a561dd4c6497173ef43e2e9ccd]). Expected version 1 got 0
I20251028 03:00:03.401455 8251 tablet_service.cc:1552] Processing DeleteTablet for tablet f39ed19223fa4268a01aaf4c7762d86e with delete_type TABLET_DATA_DELETED (replaced by tablet b6a2791098934a9c8aa21d8db88f2c9e) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.403548 8392 tablet_service.cc:1505] Processing CreateTablet for tablet 23c034566e22488b898c495c71e2eeaa (DEFAULT_TABLE table=default.Table_5b46e89d6c094f008e7da4d4ae8b6c7b [id=b1bbdd84dec6405e9dabc4e722e22734]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.403820 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23c034566e22488b898c495c71e2eeaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.406391 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet ba64524142934af08658e434e7fb08ed (table default.Table_8b7624342986474fbd0e31c86947f781 [id=530470c553154d76bf0df08ddfbf2fd0]) successfully deleted
I20251028 03:00:03.406627 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_93cfe7776e94425c91cf3b046fcd1923"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.407011 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93cfe7776e94425c91cf3b046fcd1923 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.407938 8387 tablet_service.cc:1505] Processing CreateTablet for tablet ed707eacd1b548e6afa9dd85b1a0ea81 (DEFAULT_TABLE table=default.Table_6df12dce729d47de9eb4760fec2119de [id=247f86cbdcc24cdbaaf866474a0e7013]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.407938 8122 tablet_service.cc:1505] Processing CreateTablet for tablet ed707eacd1b548e6afa9dd85b1a0ea81 (DEFAULT_TABLE table=default.Table_6df12dce729d47de9eb4760fec2119de [id=247f86cbdcc24cdbaaf866474a0e7013]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.408188 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed707eacd1b548e6afa9dd85b1a0ea81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.408260 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed707eacd1b548e6afa9dd85b1a0ea81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.410785 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 95b2dfbf834e488ea6026d0fb38cb2d1 (DEFAULT_TABLE table=default.Table_49ec85dd85624ee5bd983a6beddace4e [id=2a1e54652c67476e8ce3d751e5376059]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.411078 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b2dfbf834e488ea6026d0fb38cb2d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.411267 8251 tablet_service.cc:1505] Processing CreateTablet for tablet c90d9bc6893148b6a314f9e9036c6359 (DEFAULT_TABLE table=default.Table_5584a2f654f14984b2d85d2c44dbc692 [id=8ad209ab13e542a3a30e1790d770795c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.411470 8249 tablet_service.cc:1505] Processing CreateTablet for tablet ed707eacd1b548e6afa9dd85b1a0ea81 (DEFAULT_TABLE table=default.Table_6df12dce729d47de9eb4760fec2119de [id=247f86cbdcc24cdbaaf866474a0e7013]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.411566 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c90d9bc6893148b6a314f9e9036c6359. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.411746 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed707eacd1b548e6afa9dd85b1a0ea81. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:03.413277 7956 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet 4a57e122bbb94a9fac8c8254ab73a21f (table default.Table_19049cf1e5f2419fbff05d1df500a3ae [id=091f678d62af4a0792dfccedb3db996d], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4a57e122bbb94a9fac8c8254ab73a21f consensus state has no leader
I20251028 03:00:03.414273 8550 ts_tablet_manager.cc:1916] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.414695 8115 tablet_service.cc:1505] Processing CreateTablet for tablet 324146b2986143e997a1b9ac64df3784 (DEFAULT_TABLE table=default.Table_d949a516b2954d27b669347dac4093bf [id=e52aca571f824621b0cf2214703191aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.414808 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 56688baf397c4fbba332a5c129b505e4 (DEFAULT_TABLE table=default.Table_a1f2e79fb7f743889dce73e1979b2832 [id=a7edfdc0adcd4f5186637ffab8d669bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.414963 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 324146b2986143e997a1b9ac64df3784. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.415192 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56688baf397c4fbba332a5c129b505e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.415375 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 9b98c14370a341fd9939fceb536c8ed6 (DEFAULT_TABLE table=default.Table_e7689390da9341cc8efa3e3a38c15a47 [id=b82d9f3df7c94dd29c7426729bb32308]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.415639 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b98c14370a341fd9939fceb536c8ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.415827 8127 tablet_service.cc:1505] Processing CreateTablet for tablet c201b8bbb498417d8dd66fa05f0eb214 (DEFAULT_TABLE table=default.Table_d2e449df11e04f8fbf35620bdc11b19d [id=6114d2d9e62d479180d9b87520da39ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.416025 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c201b8bbb498417d8dd66fa05f0eb214. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.416568 8390 tablet_service.cc:1505] Processing CreateTablet for tablet f78e3987a9c6401a8d4a144f5f6dc870 (DEFAULT_TABLE table=default.Table_de56c060321c452aa62bcc0844c2f3d1 [id=f2e6b704f9e549dfbc4a1eb62396f143]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.416852 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78e3987a9c6401a8d4a144f5f6dc870. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.417555 8396 tablet_service.cc:1505] Processing CreateTablet for tablet 9ce0810477a943b9afd96053efcaa805 (DEFAULT_TABLE table=default.Table_c2771559bf9e405cb2b3d14ea5f64966 [id=4cba55a224f140bda52d46dc2643f3ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.417797 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce0810477a943b9afd96053efcaa805. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.418064 8118 tablet_service.cc:1505] Processing CreateTablet for tablet bbc32d3bc1a9489b93425ce577d9d9ae (DEFAULT_TABLE table=default.Table_3f475aaefd244e529f54110a86eb240f [id=c06be20a73be432c8b9e54d243dc9efd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.418254 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc32d3bc1a9489b93425ce577d9d9ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.419193 8400 tablet_service.cc:1505] Processing CreateTablet for tablet bbc32d3bc1a9489b93425ce577d9d9ae (DEFAULT_TABLE table=default.Table_3f475aaefd244e529f54110a86eb240f [id=c06be20a73be432c8b9e54d243dc9efd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.419459 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc32d3bc1a9489b93425ce577d9d9ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.419510 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 324146b2986143e997a1b9ac64df3784 (DEFAULT_TABLE table=default.Table_d949a516b2954d27b669347dac4093bf [id=e52aca571f824621b0cf2214703191aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.419739 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 324146b2986143e997a1b9ac64df3784. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.420372 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 9ce0810477a943b9afd96053efcaa805 (DEFAULT_TABLE table=default.Table_c2771559bf9e405cb2b3d14ea5f64966 [id=4cba55a224f140bda52d46dc2643f3ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.420583 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce0810477a943b9afd96053efcaa805. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.421407 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 782aed99e4b2455893de8896aa456ce9 (DEFAULT_TABLE table=default.Table_b487827beda64cfe9fc5cdf1daa37a4d [id=37b9f64d3a384f15a9b8e892b69e69f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.421594 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 782aed99e4b2455893de8896aa456ce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.422572 8389 tablet_service.cc:1505] Processing CreateTablet for tablet c201b8bbb498417d8dd66fa05f0eb214 (DEFAULT_TABLE table=default.Table_d2e449df11e04f8fbf35620bdc11b19d [id=6114d2d9e62d479180d9b87520da39ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.422813 8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c201b8bbb498417d8dd66fa05f0eb214. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.423659 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 782aed99e4b2455893de8896aa456ce9 (DEFAULT_TABLE table=default.Table_b487827beda64cfe9fc5cdf1daa37a4d [id=37b9f64d3a384f15a9b8e892b69e69f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.423892 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 782aed99e4b2455893de8896aa456ce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.424036 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 782aed99e4b2455893de8896aa456ce9 (DEFAULT_TABLE table=default.Table_b487827beda64cfe9fc5cdf1daa37a4d [id=37b9f64d3a384f15a9b8e892b69e69f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.424326 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 782aed99e4b2455893de8896aa456ce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.424906 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 56688baf397c4fbba332a5c129b505e4 (DEFAULT_TABLE table=default.Table_a1f2e79fb7f743889dce73e1979b2832 [id=a7edfdc0adcd4f5186637ffab8d669bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.425050 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 56688baf397c4fbba332a5c129b505e4 (DEFAULT_TABLE table=default.Table_a1f2e79fb7f743889dce73e1979b2832 [id=a7edfdc0adcd4f5186637ffab8d669bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.425113 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56688baf397c4fbba332a5c129b505e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.425283 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56688baf397c4fbba332a5c129b505e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.426126 8122 tablet_service.cc:1505] Processing CreateTablet for tablet f78e3987a9c6401a8d4a144f5f6dc870 (DEFAULT_TABLE table=default.Table_de56c060321c452aa62bcc0844c2f3d1 [id=f2e6b704f9e549dfbc4a1eb62396f143]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.426348 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78e3987a9c6401a8d4a144f5f6dc870. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.426581 8259 tablet_service.cc:1505] Processing CreateTablet for tablet f78e3987a9c6401a8d4a144f5f6dc870 (DEFAULT_TABLE table=default.Table_de56c060321c452aa62bcc0844c2f3d1 [id=f2e6b704f9e549dfbc4a1eb62396f143]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.426780 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78e3987a9c6401a8d4a144f5f6dc870. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.427223 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 9b98c14370a341fd9939fceb536c8ed6 (DEFAULT_TABLE table=default.Table_e7689390da9341cc8efa3e3a38c15a47 [id=b82d9f3df7c94dd29c7426729bb32308]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.427464 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b98c14370a341fd9939fceb536c8ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.427511 8256 tablet_service.cc:1505] Processing CreateTablet for tablet c201b8bbb498417d8dd66fa05f0eb214 (DEFAULT_TABLE table=default.Table_d2e449df11e04f8fbf35620bdc11b19d [id=6114d2d9e62d479180d9b87520da39ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.427714 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c201b8bbb498417d8dd66fa05f0eb214. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.428272 10335 raft_consensus.cc:493] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.428383 10335 raft_consensus.cc:515] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.428792 10335 leader_election.cc:290] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.429335 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0dad91f926648c781d1582733d6ae95" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:03.429450 8416 raft_consensus.cc:2468] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:03.429854 8252 tablet_service.cc:1505] Processing CreateTablet for tablet bbc32d3bc1a9489b93425ce577d9d9ae (DEFAULT_TABLE table=default.Table_3f475aaefd244e529f54110a86eb240f [id=c06be20a73be432c8b9e54d243dc9efd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.430085 8258 tablet_service.cc:1505] Processing CreateTablet for tablet da885b5494f24ec391a63ab4624e5009 (DEFAULT_TABLE table=default.Table_ad4030fee54c4132b2fba53c521bedd6 [id=7d4399fd008b4e6d8c25fa81755b5842]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.430099 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0dad91f926648c781d1582733d6ae95" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.430246 8263 tablet_service.cc:1505] Processing CreateTablet for tablet 9ce0810477a943b9afd96053efcaa805 (DEFAULT_TABLE table=default.Table_c2771559bf9e405cb2b3d14ea5f64966 [id=4cba55a224f140bda52d46dc2643f3ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.430449 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 95b2dfbf834e488ea6026d0fb38cb2d1 (DEFAULT_TABLE table=default.Table_49ec85dd85624ee5bd983a6beddace4e [id=2a1e54652c67476e8ce3d751e5376059]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.430099 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbc32d3bc1a9489b93425ce577d9d9ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.430781 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b2dfbf834e488ea6026d0fb38cb2d1. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:03.430922 8083 leader_election.cc:343] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.431052 8083 leader_election.cc:304] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:03.431550 8253 tablet_service.cc:1505] Processing CreateTablet for tablet 9b98c14370a341fd9939fceb536c8ed6 (DEFAULT_TABLE table=default.Table_e7689390da9341cc8efa3e3a38c15a47 [id=b82d9f3df7c94dd29c7426729bb32308]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.431787 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b98c14370a341fd9939fceb536c8ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.432639 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ce0810477a943b9afd96053efcaa805. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.433144 8386 tablet_service.cc:1505] Processing CreateTablet for tablet da885b5494f24ec391a63ab4624e5009 (DEFAULT_TABLE table=default.Table_ad4030fee54c4132b2fba53c521bedd6 [id=7d4399fd008b4e6d8c25fa81755b5842]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.433362 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da885b5494f24ec391a63ab4624e5009. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.436031 10335 raft_consensus.cc:2804] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:03.436105 10335 raft_consensus.cc:493] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:03.436143 10335 raft_consensus.cc:3060] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.436672 8550 ts_tablet_manager.cc:1929] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.436753 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 324146b2986143e997a1b9ac64df3784 (DEFAULT_TABLE table=default.Table_d949a516b2954d27b669347dac4093bf [id=e52aca571f824621b0cf2214703191aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.437006 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 324146b2986143e997a1b9ac64df3784. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.436923 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c01a7348790a498eaa6a628519733f77"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.437361 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c01a7348790a498eaa6a628519733f77 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.438241 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c2d77e5496bd4e578f2a583d600401cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.438562 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2d77e5496bd4e578f2a583d600401cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.439451 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_94bf2b7d93fa48b28993b26e1abdc5e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.439779 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94bf2b7d93fa48b28993b26e1abdc5e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.440280 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da885b5494f24ec391a63ab4624e5009. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.440496 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4a26d5c6c75d402c994cc1481cd8c6a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.440810 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a26d5c6c75d402c994cc1481cd8c6a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.441458 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_aa11196c63eb4ee4ba2c3597325d460a" } new_table_name: "default.Table_7cac0348a3bd4625b1daa54c5cd075b0" modify_external_catalogs: true
I20251028 03:00:03.442126 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_5b46e89d6c094f008e7da4d4ae8b6c7b" } new_table_name: "default.Table_cec8e082702e465494d62d64c6754f55" modify_external_catalogs: true
I20251028 03:00:03.442507 8392 tablet_service.cc:1505] Processing CreateTablet for tablet c90d9bc6893148b6a314f9e9036c6359 (DEFAULT_TABLE table=default.Table_5584a2f654f14984b2d85d2c44dbc692 [id=8ad209ab13e542a3a30e1790d770795c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.442901 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c90d9bc6893148b6a314f9e9036c6359. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.444413 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 9a81841c22ae49c0a17931d90407accc (table default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]). Expected version 1 got 0
I20251028 03:00:03.447654 8552 ts_tablet_manager.cc:1916] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.448074 7965 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7fb659aacea847c180ab8750d2fc1292" } new_table_name: "default.Table_b5840f20347d419087a329addb932827" modify_external_catalogs: true
I20251028 03:00:03.448881 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5a9578228e3b45b9a6d9a39b4d440b81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.449276 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a9578228e3b45b9a6d9a39b4d440b81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.450620 8552 ts_tablet_manager.cc:1929] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.450711 8552 log.cc:1199] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/8dab70ca2d9f4b5fba338cb337b0b396
I20251028 03:00:03.450912 8552 ts_tablet_manager.cc:1950] T 8dab70ca2d9f4b5fba338cb337b0b396 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.436744 8550 log.cc:1199] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8483cbc3099644d2b6f7ff52869e21ea
I20251028 03:00:03.456025 8550 ts_tablet_manager.cc:1950] T 8483cbc3099644d2b6f7ff52869e21ea P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:03.456903 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 95b2dfbf834e488ea6026d0fb38cb2d1 (DEFAULT_TABLE table=default.Table_49ec85dd85624ee5bd983a6beddace4e [id=2a1e54652c67476e8ce3d751e5376059]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.457198 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b2dfbf834e488ea6026d0fb38cb2d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.437131 10335 raft_consensus.cc:515] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.458810 10335 leader_election.cc:290] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.459473 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8483cbc3099644d2b6f7ff52869e21ea (table default.Table_8d295e2ce9374929ab925a8d8329920d [id=de05ccfb39064421868853d6d9866bda]) successfully deleted
I20251028 03:00:03.460075 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0d5bef57eadd45538258da2301f0cd31" } modify_external_catalogs: true
I20251028 03:00:03.460197 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_0d5bef57eadd45538258da2301f0cd31" } modify_external_catalogs: true
I20251028 03:00:03.462153 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_11f1aff812d14b0f8b0004ae5438d7e0" } modify_external_catalogs: true
I20251028 03:00:03.462257 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_11f1aff812d14b0f8b0004ae5438d7e0" } modify_external_catalogs: true
W20251028 03:00:03.463033 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 11f8b9bffa3048bdae44d3eaf25cf1f1 consensus state has no leader
I20251028 03:00:03.463234 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2687d010e9e444698c450ff0db40f8da" } modify_external_catalogs: true
I20251028 03:00:03.463346 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2687d010e9e444698c450ff0db40f8da" } modify_external_catalogs: true
I20251028 03:00:03.468405 8546 tablet_bootstrap.cc:492] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.468518 8546 ts_tablet_manager.cc:1403] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.098s user 0.002s sys 0.000s
I20251028 03:00:03.469079 8546 raft_consensus.cc:359] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.469208 8546 raft_consensus.cc:385] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.469245 8546 raft_consensus.cc:740] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.469373 8546 consensus_queue.cc:260] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.469681 8546 ts_tablet_manager.cc:1434] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.469928 8546 tablet_bootstrap.cc:492] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.470983 8546 tablet_bootstrap.cc:654] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.485376 9595 raft_consensus.cc:493] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.485549 9595 raft_consensus.cc:515] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.486073 9595 leader_election.cc:290] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:03.486487 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e3b33c4d189247a89878b6a8e1fcbac0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:03.486650 8416 raft_consensus.cc:2393] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in this term.
I20251028 03:00:03.487588 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e3b33c4d189247a89878b6a8e1fcbac0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:03.490991 8218 leader_election.cc:304] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.491520 8553 raft_consensus.cc:3060] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.492688 8553 raft_consensus.cc:2749] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.495011 8401 tablet_service.cc:1505] Processing CreateTablet for tablet 6f759b084a374cc2a57e861ce375d7c1 (DEFAULT_TABLE table=default.Table_5fc1ce4002024900b764d32451fe2695 [id=c2ce98fbcbff43c7981963e07b7e3f10]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.495139 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 83639e6b4b9e454ca3ce56903d2d0337 (DEFAULT_TABLE table=default.Table_f2dee7552f5543b0ac6899b7a28431a2 [id=715f093c061941709983b951959cb8c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.495369 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f759b084a374cc2a57e861ce375d7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.496197 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83639e6b4b9e454ca3ce56903d2d0337. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.497812 8397 tablet_service.cc:1505] Processing CreateTablet for tablet f6864bcffa17438eb39f90fce8acb18e (DEFAULT_TABLE table=default.Table_a27ed96bb5934fbd98f3015a38eff495 [id=7a6f632985a8419382ac2517bc9813b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.498050 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6864bcffa17438eb39f90fce8acb18e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.498530 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 8dab70ca2d9f4b5fba338cb337b0b396 (table default.Table_cff7897b7440432c96a505a493a0b149 [id=eab0e9b296d04340b7a2fd250f68d9f3]) successfully deleted
I20251028 03:00:03.499234 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 2310a46fdcb2478faf8fdc5284aa5740 (DEFAULT_TABLE table=default.Table_0325897e2eb5414685dd8be9e6d2fd2e [id=b9c19c3aa17e4059b3f58533143edff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.499498 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2310a46fdcb2478faf8fdc5284aa5740. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.500133 8389 tablet_service.cc:1505] Processing CreateTablet for tablet c99a9bd6c364463d913cfe758f3a981c (DEFAULT_TABLE table=default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.500350 8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c99a9bd6c364463d913cfe758f3a981c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.501142 8398 tablet_service.cc:1505] Processing CreateTablet for tablet 1ae522d3d5894d49b91b8b80bc89ddfc (DEFAULT_TABLE table=default.Table_2b2a784556c14e418b568477963b0814 [id=9e39385085c84e5fb2bbc07427752c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.501364 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ae522d3d5894d49b91b8b80bc89ddfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.502060 8400 tablet_service.cc:1505] Processing CreateTablet for tablet 1645b5cb20584f108b0dc46379c32cde (DEFAULT_TABLE table=default.Table_c676d258d54548aa89e8cc87d30171a6 [id=65d772ae61fc4e5191b8b5f346899d5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.502409 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 5e649541935b458f8e1583c2db9e23f5 (DEFAULT_TABLE table=default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.495330 8396 tablet_service.cc:1505] Processing CreateTablet for tablet a077a4dae64547669baf0590795ba98e (DEFAULT_TABLE table=default.Table_f88362a62f9d434b9bbf51c049572ae1 [id=cbf251bcb4254132b41be09798361f97]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.502692 8401 tablet_service.cc:1505] Processing CreateTablet for tablet ad93dc540ecb47ecb3959d20b1e3dfd9 (DEFAULT_TABLE table=default.Table_42b51970061d47558809ca3152c09f84 [id=0d8d75f85af74f35a5e8bafee848bab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.502950 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e649541935b458f8e1583c2db9e23f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.503907 8251 tablet_service.cc:1505] Processing CreateTablet for tablet 94a43fc2b1694823919e33c2dafa5037 (DEFAULT_TABLE table=default.Table_c73a1345f4bd40e3b39ca5c4f3cd4fc2 [id=b3f8e98592fa43ebbcc874e8b0be6c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.503954 8249 tablet_service.cc:1505] Processing CreateTablet for tablet 67c0d8776ff94da1a2249d5a288568da (DEFAULT_TABLE table=default.Table_008b1b0c028d4fdface45a3b2110508d [id=b8ae9b94370c4b3ba1ac2af86c4762d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.504191 8253 tablet_service.cc:1505] Processing CreateTablet for tablet c99a9bd6c364463d913cfe758f3a981c (DEFAULT_TABLE table=default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.504220 8259 tablet_service.cc:1505] Processing CreateTablet for tablet 6f759b084a374cc2a57e861ce375d7c1 (DEFAULT_TABLE table=default.Table_5fc1ce4002024900b764d32451fe2695 [id=c2ce98fbcbff43c7981963e07b7e3f10]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.504462 8263 tablet_service.cc:1505] Processing CreateTablet for tablet f6864bcffa17438eb39f90fce8acb18e (DEFAULT_TABLE table=default.Table_a27ed96bb5934fbd98f3015a38eff495 [id=7a6f632985a8419382ac2517bc9813b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.504534 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 26545f3a4b1c4a21a223fa258a490e08 (DEFAULT_TABLE table=default.Table_2a5eacfbc4da4f6ebe5efca52a273f78 [id=386b91efce104784aa519cd57c46f8f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.504696 8252 tablet_service.cc:1505] Processing CreateTablet for tablet a077a4dae64547669baf0590795ba98e (DEFAULT_TABLE table=default.Table_f88362a62f9d434b9bbf51c049572ae1 [id=cbf251bcb4254132b41be09798361f97]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.504971 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 83639e6b4b9e454ca3ce56903d2d0337 (DEFAULT_TABLE table=default.Table_f2dee7552f5543b0ac6899b7a28431a2 [id=715f093c061941709983b951959cb8c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.505422 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 1ae522d3d5894d49b91b8b80bc89ddfc (DEFAULT_TABLE table=default.Table_2b2a784556c14e418b568477963b0814 [id=9e39385085c84e5fb2bbc07427752c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.505481 8257 tablet_service.cc:1505] Processing CreateTablet for tablet ad93dc540ecb47ecb3959d20b1e3dfd9 (DEFAULT_TABLE table=default.Table_42b51970061d47558809ca3152c09f84 [id=0d8d75f85af74f35a5e8bafee848bab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.505934 9595 raft_consensus.cc:493] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.506029 9595 raft_consensus.cc:515] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.506346 8265 tablet_service.cc:1505] Processing CreateTablet for tablet 2310a46fdcb2478faf8fdc5284aa5740 (DEFAULT_TABLE table=default.Table_0325897e2eb5414685dd8be9e6d2fd2e [id=b9c19c3aa17e4059b3f58533143edff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.506454 9595 leader_election.cc:290] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:03.506592 8247 tablet_service.cc:1505] Processing CreateTablet for tablet 1645b5cb20584f108b0dc46379c32cde (DEFAULT_TABLE table=default.Table_c676d258d54548aa89e8cc87d30171a6 [id=65d772ae61fc4e5191b8b5f346899d5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.506825 8255 tablet_service.cc:1505] Processing CreateTablet for tablet 5e649541935b458f8e1583c2db9e23f5 (DEFAULT_TABLE table=default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.507616 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ad926e9ee0044c62a7abbfca959a4f56" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:03.509687 8546 tablet_bootstrap.cc:492] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.509820 8546 ts_tablet_manager.cc:1403] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.000s
I20251028 03:00:03.510337 8546 raft_consensus.cc:359] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.510445 8546 raft_consensus.cc:385] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.510480 8546 raft_consensus.cc:740] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.510596 8546 consensus_queue.cc:260] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.510900 8546 ts_tablet_manager.cc:1434] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.511430 10242 raft_consensus.cc:493] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.511538 10242 raft_consensus.cc:515] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.511955 10242 leader_election.cc:290] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.512200 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ad926e9ee0044c62a7abbfca959a4f56" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:03.512548 8401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad93dc540ecb47ecb3959d20b1e3dfd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.469095 8552 tablet_replica.cc:333] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.512673 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0dad91f926648c781d1582733d6ae95" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:03.512764 8416 raft_consensus.cc:3060] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.512785 8552 raft_consensus.cc:2243] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.512884 8552 raft_consensus.cc:2272] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.513237 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 67c0d8776ff94da1a2249d5a288568da (DEFAULT_TABLE table=default.Table_008b1b0c028d4fdface45a3b2110508d [id=b8ae9b94370c4b3ba1ac2af86c4762d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.513516 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67c0d8776ff94da1a2249d5a288568da. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.513856 8416 raft_consensus.cc:2468] T d0dad91f926648c781d1582733d6ae95 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:03.514413 8251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94a43fc2b1694823919e33c2dafa5037. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.515040 8400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1645b5cb20584f108b0dc46379c32cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.515563 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0dad91f926648c781d1582733d6ae95" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:03.516446 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 6f759b084a374cc2a57e861ce375d7c1 (DEFAULT_TABLE table=default.Table_5fc1ce4002024900b764d32451fe2695 [id=c2ce98fbcbff43c7981963e07b7e3f10]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.516690 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f759b084a374cc2a57e861ce375d7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.517268 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a077a4dae64547669baf0590795ba98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.518793 8118 tablet_service.cc:1505] Processing CreateTablet for tablet f6864bcffa17438eb39f90fce8acb18e (DEFAULT_TABLE table=default.Table_a27ed96bb5934fbd98f3015a38eff495 [id=7a6f632985a8419382ac2517bc9813b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.519024 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6864bcffa17438eb39f90fce8acb18e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.519663 8220 leader_election.cc:304] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.520057 9595 raft_consensus.cc:2749] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.520380 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f62b981b9b1a44bb96bbc90f924c3b64" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:03.520490 8148 raft_consensus.cc:2468] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:03.520784 8354 leader_election.cc:304] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:03.521162 10242 raft_consensus.cc:2804] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:03.521240 10242 raft_consensus.cc:493] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:03.521289 10242 raft_consensus.cc:3060] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.521492 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 9c86c3122a74451e9685d79474bda1f6 (DEFAULT_TABLE table=default.Table_73757725ad844037851210da1a02399a [id=5dbc7d7bd9ba4c0db9b5d39b38779c99]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.521595 8115 tablet_service.cc:1505] Processing CreateTablet for tablet 83639e6b4b9e454ca3ce56903d2d0337 (DEFAULT_TABLE table=default.Table_f2dee7552f5543b0ac6899b7a28431a2 [id=715f093c061941709983b951959cb8c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.521848 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83639e6b4b9e454ca3ce56903d2d0337. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.521998 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 2310a46fdcb2478faf8fdc5284aa5740 (DEFAULT_TABLE table=default.Table_0325897e2eb5414685dd8be9e6d2fd2e [id=b9c19c3aa17e4059b3f58533143edff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.522094 8116 tablet_service.cc:1505] Processing CreateTablet for tablet 5e649541935b458f8e1583c2db9e23f5 (DEFAULT_TABLE table=default.Table_e067b14c102644fdb1c3a0bbaa79746a [id=3437f62f738b4ab8b978437e038d8375]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.522224 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2310a46fdcb2478faf8fdc5284aa5740. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.522338 8116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e649541935b458f8e1583c2db9e23f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.522514 8117 tablet_service.cc:1505] Processing CreateTablet for tablet 1645b5cb20584f108b0dc46379c32cde (DEFAULT_TABLE table=default.Table_c676d258d54548aa89e8cc87d30171a6 [id=65d772ae61fc4e5191b8b5f346899d5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.446413 10253 consensus_queue.cc:1048] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.076s
I20251028 03:00:03.522627 8113 tablet_service.cc:1505] Processing CreateTablet for tablet 1ae522d3d5894d49b91b8b80bc89ddfc (DEFAULT_TABLE table=default.Table_2b2a784556c14e418b568477963b0814 [id=9e39385085c84e5fb2bbc07427752c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.522724 8117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1645b5cb20584f108b0dc46379c32cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.522863 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ae522d3d5894d49b91b8b80bc89ddfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.523015 8124 tablet_service.cc:1505] Processing CreateTablet for tablet ad93dc540ecb47ecb3959d20b1e3dfd9 (DEFAULT_TABLE table=default.Table_42b51970061d47558809ca3152c09f84 [id=0d8d75f85af74f35a5e8bafee848bab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.523200 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad93dc540ecb47ecb3959d20b1e3dfd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.521600 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 26545f3a4b1c4a21a223fa258a490e08 (DEFAULT_TABLE table=default.Table_2a5eacfbc4da4f6ebe5efca52a273f78 [id=386b91efce104784aa519cd57c46f8f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.524569 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26545f3a4b1c4a21a223fa258a490e08. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.522329 10242 raft_consensus.cc:515] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.526970 10242 leader_election.cc:290] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.526986 8118 tablet_service.cc:1505] Processing CreateTablet for tablet c99a9bd6c364463d913cfe758f3a981c (DEFAULT_TABLE table=default.Table_28e48d9cf44e45a8bd74ab8ef2bb0d93 [id=f9451803c95e447ba6d91ba97c1b6995]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.527068 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f62b981b9b1a44bb96bbc90f924c3b64" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.527194 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c99a9bd6c364463d913cfe758f3a981c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.521826 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 26545f3a4b1c4a21a223fa258a490e08 (DEFAULT_TABLE table=default.Table_2a5eacfbc4da4f6ebe5efca52a273f78 [id=386b91efce104784aa519cd57c46f8f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.527563 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26545f3a4b1c4a21a223fa258a490e08. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.528954 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f62b981b9b1a44bb96bbc90f924c3b64" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:03.529071 8148 raft_consensus.cc:3060] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.529968 8148 raft_consensus.cc:2468] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:03.530288 8354 leader_election.cc:304] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:03.530455 10242 raft_consensus.cc:2804] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:03.530537 10242 raft_consensus.cc:697] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:03.530676 10242 consensus_queue.cc:237] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.533144 8120 tablet_service.cc:1505] Processing CreateTablet for tablet a077a4dae64547669baf0590795ba98e (DEFAULT_TABLE table=default.Table_f88362a62f9d434b9bbf51c049572ae1 [id=cbf251bcb4254132b41be09798361f97]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.533429 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a077a4dae64547669baf0590795ba98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.534381 8253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c99a9bd6c364463d913cfe758f3a981c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.534420 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 9c86c3122a74451e9685d79474bda1f6 (DEFAULT_TABLE table=default.Table_73757725ad844037851210da1a02399a [id=5dbc7d7bd9ba4c0db9b5d39b38779c99]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.534667 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c86c3122a74451e9685d79474bda1f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.536371 8552 ts_tablet_manager.cc:1916] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.537467 8259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f759b084a374cc2a57e861ce375d7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.538722 8552 ts_tablet_manager.cc:1929] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.539669 8263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6864bcffa17438eb39f90fce8acb18e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.543838 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e649541935b458f8e1583c2db9e23f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.544095 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a077a4dae64547669baf0590795ba98e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.544207 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26545f3a4b1c4a21a223fa258a490e08. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.544348 8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1645b5cb20584f108b0dc46379c32cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.546018 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83639e6b4b9e454ca3ce56903d2d0337. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.541278 8546 tablet_bootstrap.cc:492] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.547387 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67c0d8776ff94da1a2249d5a288568da. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.547935 8546 tablet_bootstrap.cc:654] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.548128 10253 raft_consensus.cc:493] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.548221 10253 raft_consensus.cc:515] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.548641 10253 leader_election.cc:290] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.548805 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c86c3122a74451e9685d79474bda1f6. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:03.548822 8083 leader_election.cc:343] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.548942 8083 leader_election.cc:304] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:03.549091 10253 raft_consensus.cc:2804] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:03.549170 10253 raft_consensus.cc:697] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:03.549311 10253 consensus_queue.cc:237] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.549381 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad93dc540ecb47ecb3959d20b1e3dfd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.530032 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 67c0d8776ff94da1a2249d5a288568da (DEFAULT_TABLE table=default.Table_008b1b0c028d4fdface45a3b2110508d [id=b8ae9b94370c4b3ba1ac2af86c4762d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.551410 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ae522d3d5894d49b91b8b80bc89ddfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.551450 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67c0d8776ff94da1a2249d5a288568da. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.553603 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2310a46fdcb2478faf8fdc5284aa5740. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.554822 8273 raft_consensus.cc:3060] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.555444 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6818c3f2dfa48448bb2d9b38ed0dc19" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.556699 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6818c3f2dfa48448bb2d9b38ed0dc19" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:03.556861 8419 raft_consensus.cc:2468] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:03.557256 10187 tablet.cc:1722] T ad926e9ee0044c62a7abbfca959a4f56 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:03.558601 8392 tablet_service.cc:1505] Processing CreateTablet for tablet 94a43fc2b1694823919e33c2dafa5037 (DEFAULT_TABLE table=default.Table_c73a1345f4bd40e3b39ca5c4f3cd4fc2 [id=b3f8e98592fa43ebbcc874e8b0be6c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.558693 8083 leader_election.cc:304] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:03.558892 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94a43fc2b1694823919e33c2dafa5037. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.559051 10335 raft_consensus.cc:2804] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:03.559101 10335 raft_consensus.cc:493] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:03.559139 10335 raft_consensus.cc:3060] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.437898 8111 tablet_service.cc:1505] Processing CreateTablet for tablet da885b5494f24ec391a63ab4624e5009 (DEFAULT_TABLE table=default.Table_ad4030fee54c4132b2fba53c521bedd6 [id=7d4399fd008b4e6d8c25fa81755b5842]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.559787 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da885b5494f24ec391a63ab4624e5009. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.560096 10335 raft_consensus.cc:515] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.560453 10335 leader_election.cc:290] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:03.560796 8083 leader_election.cc:343] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.561364 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6818c3f2dfa48448bb2d9b38ed0dc19" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:03.561398 8282 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6818c3f2dfa48448bb2d9b38ed0dc19" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:03.561478 8419 raft_consensus.cc:3060] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:03.561841 7955 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.154.130:43525, user_credentials={real_user=slave}} blocked reactor thread for 63311.3us
I20251028 03:00:03.562005 8552 log.cc:1199] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/f39ed19223fa4268a01aaf4c7762d86e
I20251028 03:00:03.562367 8552 ts_tablet_manager.cc:1950] T f39ed19223fa4268a01aaf4c7762d86e P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.562389 8419 raft_consensus.cc:2468] T f6818c3f2dfa48448bb2d9b38ed0dc19 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
W20251028 03:00:03.510918 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23c034566e22488b898c495c71e2eeaa consensus state has no leader
I20251028 03:00:03.562923 8083 leader_election.cc:304] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:03.562893 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0e2fb96fdd1e49429c3f2e5dc91bf766"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.563395 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0e2fb96fdd1e49429c3f2e5dc91bf766 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:03.511085 7965 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a279d68990284332bae453e4d26f1036 consensus state has no leader
I20251028 03:00:03.564361 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_dfa4ca07c0c54cef8189b4c26ac43c9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.564703 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dfa4ca07c0c54cef8189b4c26ac43c9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.565410 8551 ts_tablet_manager.cc:1916] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.565486 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4dc821cafb8143f2aff8dab1d512a544"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.565819 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4dc821cafb8143f2aff8dab1d512a544 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.566524 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ff74a0ebc6284f5f813ca17bd15504d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.566802 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff74a0ebc6284f5f813ca17bd15504d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.567431 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1562d90171b44a268e4777c51b3c8483" } modify_external_catalogs: true
I20251028 03:00:03.567517 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1562d90171b44a268e4777c51b3c8483" } modify_external_catalogs: true
I20251028 03:00:03.536473 7963 catalog_manager.cc:5649] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 03:00:03.554361 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5c1994484abf4b6fbbc3f47527558cdb
I20251028 03:00:03.569358 8546 tablet_bootstrap.cc:492] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.569451 8546 ts_tablet_manager.cc:1403] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.002s
I20251028 03:00:03.569669 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3355092ddb794533a5545c15b2847d15" } new_table_name: "default.Table_df2fde6b445941dbbb064a8563b22f13" modify_external_catalogs: true
I20251028 03:00:03.570015 8546 raft_consensus.cc:359] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.570122 8546 raft_consensus.cc:385] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.570156 8546 raft_consensus.cc:740] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.570307 8546 consensus_queue.cc:260] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.570473 7963 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet f62b981b9b1a44bb96bbc90f924c3b64 (table default.Table_3f85bca98f84492da9cc6b47c772334d [id=d4eaebd005bc42dd86d7125ef555546f]). Expected version 5 got 0
I20251028 03:00:03.570698 8546 ts_tablet_manager.cc:1434] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:03.554544 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet bbdd209e156c482a8910379e73283f4b
I20251028 03:00:03.570902 8546 tablet_bootstrap.cc:492] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.571168 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e067b14c102644fdb1c3a0bbaa79746a" } new_table_name: "default.Table_f2d7698ebaad442093545d248e08bb21" modify_external_catalogs: true
I20251028 03:00:03.554662 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet fcab55fd49da41f7a95d46d76a8dac8b
I20251028 03:00:03.572104 10335 raft_consensus.cc:2804] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:03.572149 8546 tablet_bootstrap.cc:654] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.572297 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_49f2ad3b1c934eaeab556927604ddf1b" } modify_external_catalogs: true
I20251028 03:00:03.572391 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_49f2ad3b1c934eaeab556927604ddf1b" } modify_external_catalogs: true
I20251028 03:00:03.572506 10335 raft_consensus.cc:697] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:03.572664 10335 consensus_queue.cc:237] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:03.574642 8083 leader_election.cc:343] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.576483 8546 tablet_bootstrap.cc:492] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.576658 8546 ts_tablet_manager.cc:1403] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 03:00:03.577216 8546 raft_consensus.cc:359] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.577384 8546 raft_consensus.cc:385] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.577673 8546 raft_consensus.cc:740] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.577883 8546 consensus_queue.cc:260] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.578258 8546 ts_tablet_manager.cc:1434] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.580111 8546 tablet_bootstrap.cc:492] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.581840 8546 tablet_bootstrap.cc:654] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.583506 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet f39ed19223fa4268a01aaf4c7762d86e (table default.Table_4debdeef26b14d9b93a878210eaba4c3 [id=5a391450a3a04f01bcd5c3c06d36cf66]) successfully deleted
I20251028 03:00:03.584304 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 9c86c3122a74451e9685d79474bda1f6 (DEFAULT_TABLE table=default.Table_73757725ad844037851210da1a02399a [id=5dbc7d7bd9ba4c0db9b5d39b38779c99]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.584532 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet bbdd209e156c482a8910379e73283f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.584681 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c86c3122a74451e9685d79474bda1f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.584692 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet fcab55fd49da41f7a95d46d76a8dac8b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.581799 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet 5c1994484abf4b6fbbc3f47527558cdb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.585011 8552 tablet_replica.cc:333] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.585163 8552 raft_consensus.cc:2243] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:03.585368 8552 raft_consensus.cc:2272] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.585757 8552 ts_tablet_manager.cc:1916] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.588678 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet fcab55fd49da41f7a95d46d76a8dac8b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.588778 8552 ts_tablet_manager.cc:1929] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:03.588848 8552 log.cc:1199] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/fcab55fd49da41f7a95d46d76a8dac8b
I20251028 03:00:03.588865 8550 tablet_replica.cc:333] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:03.589015 8550 raft_consensus.cc:2243] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.589156 8550 raft_consensus.cc:2272] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.589195 8552 ts_tablet_manager.cc:1950] T fcab55fd49da41f7a95d46d76a8dac8b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.589560 8550 ts_tablet_manager.cc:1916] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.590277 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 94a43fc2b1694823919e33c2dafa5037 (DEFAULT_TABLE table=default.Table_c73a1345f4bd40e3b39ca5c4f3cd4fc2 [id=b3f8e98592fa43ebbcc874e8b0be6c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.590526 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94a43fc2b1694823919e33c2dafa5037. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.590530 8552 tablet_replica.cc:333] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.590656 8552 raft_consensus.cc:2243] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.590754 8552 raft_consensus.cc:2272] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.591068 8551 ts_tablet_manager.cc:1929] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.591136 8551 log.cc:1199] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/28b81f0000c4407096feff773c29579d
I20251028 03:00:03.591351 8551 ts_tablet_manager.cc:1950] T 28b81f0000c4407096feff773c29579d P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.591857 8550 ts_tablet_manager.cc:1929] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:03.591918 8550 log.cc:1199] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/fcab55fd49da41f7a95d46d76a8dac8b
W20251028 03:00:03.592104 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet bbdd209e156c482a8910379e73283f4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bbdd209e156c482a8910379e73283f4b already in progress: creating tablet
I20251028 03:00:03.592175 8550 ts_tablet_manager.cc:1950] T fcab55fd49da41f7a95d46d76a8dac8b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:03.592264 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet fcab55fd49da41f7a95d46d76a8dac8b (table default.Table_11f1aff812d14b0f8b0004ae5438d7e0 [id=86e0c4d129fc42f1998b2b0f2dbc2135]) successfully deleted
I20251028 03:00:03.592983 8545 tablet_bootstrap.cc:492] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.593066 8545 ts_tablet_manager.cc:1403] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.237s user 0.002s sys 0.000s
I20251028 03:00:03.593286 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet bbdd209e156c482a8910379e73283f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.593554 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 5c1994484abf4b6fbbc3f47527558cdb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.593513 8545 raft_consensus.cc:359] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.593613 8545 raft_consensus.cc:385] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.593642 8545 raft_consensus.cc:740] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.593756 8545 consensus_queue.cc:260] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.593828 8550 tablet_replica.cc:333] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:03.593966 8550 raft_consensus.cc:2243] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:03.594054 8545 ts_tablet_manager.cc:1434] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.594163 8550 raft_consensus.cc:2272] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.594237 8545 tablet_bootstrap.cc:492] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.595180 8545 tablet_bootstrap.cc:654] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.597782 8550 ts_tablet_manager.cc:1916] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.597859 8544 tablet_bootstrap.cc:492] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.597935 8544 ts_tablet_manager.cc:1403] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.218s user 0.002s sys 0.000s
I20251028 03:00:03.598311 8544 raft_consensus.cc:359] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.598405 8544 raft_consensus.cc:385] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.598435 8544 raft_consensus.cc:740] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.598544 8544 consensus_queue.cc:260] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.598807 8544 ts_tablet_manager.cc:1434] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.598981 8544 tablet_bootstrap.cc:492] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.599859 8544 tablet_bootstrap.cc:654] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.599866 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet fcab55fd49da41f7a95d46d76a8dac8b (table default.Table_11f1aff812d14b0f8b0004ae5438d7e0 [id=86e0c4d129fc42f1998b2b0f2dbc2135]) successfully deleted
W20251028 03:00:03.600117 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet bbdd209e156c482a8910379e73283f4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bbdd209e156c482a8910379e73283f4b already in progress: creating tablet
I20251028 03:00:03.600245 8550 ts_tablet_manager.cc:1929] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:03.600304 8550 log.cc:1199] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/5c1994484abf4b6fbbc3f47527558cdb
I20251028 03:00:03.600548 8550 ts_tablet_manager.cc:1950] T 5c1994484abf4b6fbbc3f47527558cdb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:03.604019 8552 ts_tablet_manager.cc:1916] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.605099 8545 tablet_bootstrap.cc:492] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.605196 8545 ts_tablet_manager.cc:1403] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 03:00:03.605785 8545 raft_consensus.cc:359] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.605901 8545 raft_consensus.cc:385] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.605937 8545 raft_consensus.cc:740] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.606037 8545 consensus_queue.cc:260] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.606324 8545 ts_tablet_manager.cc:1434] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.606503 8545 tablet_bootstrap.cc:492] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.607537 8545 tablet_bootstrap.cc:654] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.611815 8552 ts_tablet_manager.cc:1929] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:03.612902 8552 log.cc:1199] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/5c1994484abf4b6fbbc3f47527558cdb
I20251028 03:00:03.613462 8552 ts_tablet_manager.cc:1950] T 5c1994484abf4b6fbbc3f47527558cdb P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.618614 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 5c1994484abf4b6fbbc3f47527558cdb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:03.618690 8356 leader_election.cc:343] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.618762 8545 tablet_bootstrap.cc:492] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.618808 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet fcab55fd49da41f7a95d46d76a8dac8b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.618917 8545 ts_tablet_manager.cc:1403] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251028 03:00:03.619190 10245 raft_consensus.cc:493] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.619365 10245 raft_consensus.cc:515] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.619629 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 28b81f0000c4407096feff773c29579d (table default.Table_fa28ec70564a473fb82a04887b59f553 [id=682408eb6dfa4bc28bd0c7b9175a4c09]) successfully deleted
I20251028 03:00:03.619735 10245 leader_election.cc:290] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.619845 8551 tablet_replica.cc:333] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.620028 8551 raft_consensus.cc:2243] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.620157 8551 raft_consensus.cc:2272] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.618921 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet bbdd209e156c482a8910379e73283f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.620473 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 5c1994484abf4b6fbbc3f47527558cdb (table default.Table_0d5bef57eadd45538258da2301f0cd31 [id=b0407c8f78a84245a954c67e888da3ec]) successfully deleted
I20251028 03:00:03.620837 8545 raft_consensus.cc:359] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.620945 8545 raft_consensus.cc:385] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.620980 8545 raft_consensus.cc:740] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.621135 8545 consensus_queue.cc:260] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.621444 8545 ts_tablet_manager.cc:1434] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.621641 8545 tablet_bootstrap.cc:492] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.621830 8551 ts_tablet_manager.cc:1916] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.624302 8551 ts_tablet_manager.cc:1929] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:03.627079 9595 raft_consensus.cc:493] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.627189 9595 raft_consensus.cc:515] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.628939 9595 leader_election.cc:290] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:03.629352 8551 log.cc:1199] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/5c1994484abf4b6fbbc3f47527558cdb
I20251028 03:00:03.629776 8551 ts_tablet_manager.cc:1950] T 5c1994484abf4b6fbbc3f47527558cdb P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.630106 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46ae536d269d4cbdbcee44a26d25bd67" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:03.630244 8419 raft_consensus.cc:2468] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:03.630550 8551 tablet_replica.cc:333] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.630554 8220 leader_election.cc:304] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:03.630731 8551 raft_consensus.cc:2243] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.630908 9595 raft_consensus.cc:2804] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:03.630970 9595 raft_consensus.cc:493] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:03.631006 9595 raft_consensus.cc:3060] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.631026 8551 raft_consensus.cc:2272] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.631420 8551 ts_tablet_manager.cc:1916] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.632146 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 5c1994484abf4b6fbbc3f47527558cdb (table default.Table_0d5bef57eadd45538258da2301f0cd31 [id=b0407c8f78a84245a954c67e888da3ec]) successfully deleted
I20251028 03:00:03.632310 9595 raft_consensus.cc:515] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.632683 9595 leader_election.cc:290] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:03.633213 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46ae536d269d4cbdbcee44a26d25bd67" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:03.633496 8419 raft_consensus.cc:3060] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.633728 8551 ts_tablet_manager.cc:1929] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:03.635017 8551 log.cc:1199] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/fcab55fd49da41f7a95d46d76a8dac8b
I20251028 03:00:03.636679 8551 ts_tablet_manager.cc:1950] T fcab55fd49da41f7a95d46d76a8dac8b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.636795 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46ae536d269d4cbdbcee44a26d25bd67" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:03.634487 8419 raft_consensus.cc:2468] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:03.637352 8220 leader_election.cc:304] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:03.637444 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet fcab55fd49da41f7a95d46d76a8dac8b (table default.Table_11f1aff812d14b0f8b0004ae5438d7e0 [id=86e0c4d129fc42f1998b2b0f2dbc2135]) successfully deleted
I20251028 03:00:03.637544 9595 raft_consensus.cc:2804] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:03.637622 9595 raft_consensus.cc:697] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
W20251028 03:00:03.637660 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet bbdd209e156c482a8910379e73283f4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bbdd209e156c482a8910379e73283f4b already in progress: creating tablet
I20251028 03:00:03.637763 9595 consensus_queue.cc:237] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.637849 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46ae536d269d4cbdbcee44a26d25bd67" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:03.638031 8218 leader_election.cc:343] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.638351 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46ae536d269d4cbdbcee44a26d25bd67" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 03:00:03.639660 8354 leader_election.cc:343] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.640005 8275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f62b981b9b1a44bb96bbc90f924c3b64" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:03.640231 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46ae536d269d4cbdbcee44a26d25bd67" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:03.640003 8218 leader_election.cc:343] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.642138 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet fb59e5f7bb57488ba2f9e4b43a118bf8 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:03.644135 8544 tablet_bootstrap.cc:492] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.644235 8544 ts_tablet_manager.cc:1403] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.000s
I20251028 03:00:03.644367 8546 tablet_bootstrap.cc:492] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.644451 8546 ts_tablet_manager.cc:1403] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.066s user 0.002s sys 0.001s
I20251028 03:00:03.644734 8544 raft_consensus.cc:359] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.644862 8544 raft_consensus.cc:385] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.644892 8544 raft_consensus.cc:740] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.644893 8546 raft_consensus.cc:359] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.644987 8546 raft_consensus.cc:385] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.645018 8546 raft_consensus.cc:740] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.645010 8544 consensus_queue.cc:260] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.645128 8546 consensus_queue.cc:260] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.645319 8544 ts_tablet_manager.cc:1434] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.645416 8546 ts_tablet_manager.cc:1434] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.645484 8544 tablet_bootstrap.cc:492] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.645853 9190 raft_consensus.cc:493] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.645943 9190 raft_consensus.cc:515] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.646317 9190 leader_election.cc:290] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.646451 8544 tablet_bootstrap.cc:654] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:03.648288 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e3b35a5a341641738a9e6a719d6baa35 consensus state has no leader
I20251028 03:00:03.648494 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 4a57e122bbb94a9fac8c8254ab73a21f from {username='slave'} at 127.0.0.1:47364
W20251028 03:00:03.651536 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5e649541935b458f8e1583c2db9e23f5 consensus state has no leader
W20251028 03:00:03.652549 8356 leader_election.cc:343] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.653599 8356 leader_election.cc:304] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.653932 10242 raft_consensus.cc:2749] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.654247 8545 tablet_bootstrap.cc:654] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.655056 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58268ac590f3437a942dcafd927a5496" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:03.655444 8356 leader_election.cc:343] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.656776 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58268ac590f3437a942dcafd927a5496" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:03.657436 8354 leader_election.cc:343] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.657510 8354 leader_election.cc:304] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.657780 9190 raft_consensus.cc:2749] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.661175 8544 tablet_bootstrap.cc:492] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.661284 8544 ts_tablet_manager.cc:1403] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 03:00:03.661749 8544 raft_consensus.cc:359] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.661870 8544 raft_consensus.cc:385] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.661896 8544 raft_consensus.cc:740] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.662016 8544 consensus_queue.cc:260] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.662300 8544 ts_tablet_manager.cc:1434] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.662472 8544 tablet_bootstrap.cc:492] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.663484 8546 tablet_bootstrap.cc:492] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.664538 8546 tablet_bootstrap.cc:654] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.665421 8544 tablet_bootstrap.cc:654] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.665499 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 5c1994484abf4b6fbbc3f47527558cdb (table default.Table_0d5bef57eadd45538258da2301f0cd31 [id=b0407c8f78a84245a954c67e888da3ec]) successfully deleted
I20251028 03:00:03.668640 8545 tablet_bootstrap.cc:492] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.668747 8545 ts_tablet_manager.cc:1403] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.000s
I20251028 03:00:03.669245 8545 raft_consensus.cc:359] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.669346 8545 raft_consensus.cc:385] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.669377 8545 raft_consensus.cc:740] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.669494 8545 consensus_queue.cc:260] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.669808 8545 ts_tablet_manager.cc:1434] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.670024 8545 tablet_bootstrap.cc:492] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.670981 8545 tablet_bootstrap.cc:654] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.677551 8546 tablet_bootstrap.cc:492] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.677649 8545 tablet_bootstrap.cc:492] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.677691 8546 ts_tablet_manager.cc:1403] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20251028 03:00:03.677739 8545 ts_tablet_manager.cc:1403] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 03:00:03.678182 8545 raft_consensus.cc:359] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.678280 8545 raft_consensus.cc:385] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.678208 8546 raft_consensus.cc:359] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.678313 8545 raft_consensus.cc:740] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.678319 8546 raft_consensus.cc:385] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.678361 8546 raft_consensus.cc:740] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.678426 8545 consensus_queue.cc:260] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.678479 8546 consensus_queue.cc:260] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.678701 8545 ts_tablet_manager.cc:1434] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.678755 8546 ts_tablet_manager.cc:1434] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.678889 8545 tablet_bootstrap.cc:492] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.678922 8546 tablet_bootstrap.cc:492] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.679587 8553 raft_consensus.cc:493] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.679668 8553 raft_consensus.cc:515] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.679831 8545 tablet_bootstrap.cc:654] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.680006 8553 leader_election.cc:290] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:03.680953 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e45fe8888fbc415ea637b363dccdd06b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:03.681255 8546 tablet_bootstrap.cc:654] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.682283 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e45fe8888fbc415ea637b363dccdd06b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:03.682687 8544 tablet_bootstrap.cc:492] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.682778 8544 ts_tablet_manager.cc:1403] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.002s
I20251028 03:00:03.683230 8544 raft_consensus.cc:359] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.683349 8544 raft_consensus.cc:385] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.683382 8544 raft_consensus.cc:740] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.683492 8544 consensus_queue.cc:260] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.683761 8544 ts_tablet_manager.cc:1434] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251028 03:00:03.683825 8220 leader_election.cc:304] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.683955 8544 tablet_bootstrap.cc:492] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.684057 8553 raft_consensus.cc:2749] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.684917 8544 tablet_bootstrap.cc:654] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.688450 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet fb59e5f7bb57488ba2f9e4b43a118bf8
I20251028 03:00:03.688771 7965 catalog_manager.cc:6719] ReplaceTablet: tablet fb59e5f7bb57488ba2f9e4b43a118bf8 deleted and replaced by tablet b53ba38b16c84b199490ce8b93316b85
I20251028 03:00:03.691582 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet fb59e5f7bb57488ba2f9e4b43a118bf8 with delete_type TABLET_DATA_DELETED (replaced by tablet b53ba38b16c84b199490ce8b93316b85) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.691879 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet fb59e5f7bb57488ba2f9e4b43a118bf8 with delete_type TABLET_DATA_DELETED (replaced by tablet b53ba38b16c84b199490ce8b93316b85) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.692270 8544 tablet_bootstrap.cc:492] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
W20251028 03:00:03.692745 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet fb59e5f7bb57488ba2f9e4b43a118bf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fb59e5f7bb57488ba2f9e4b43a118bf8 already in progress: creating tablet
W20251028 03:00:03.692914 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet fb59e5f7bb57488ba2f9e4b43a118bf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fb59e5f7bb57488ba2f9e4b43a118bf8 already in progress: creating tablet
I20251028 03:00:03.692957 8544 ts_tablet_manager.cc:1403] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20251028 03:00:03.693899 8553 raft_consensus.cc:493] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.694002 8553 raft_consensus.cc:515] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.694365 8553 leader_election.cc:290] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:03.695262 8544 raft_consensus.cc:359] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.695451 8544 raft_consensus.cc:385] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.695523 8544 raft_consensus.cc:740] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.695726 8544 consensus_queue.cc:260] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.696051 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a38ef10c145a4d00a26265a9dee5c744" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:03.696187 8419 raft_consensus.cc:2468] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:03.696420 8220 leader_election.cc:304] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:03.696568 8553 raft_consensus.cc:2804] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:03.696604 8553 raft_consensus.cc:493] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:03.696636 8553 raft_consensus.cc:3060] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.697644 8553 raft_consensus.cc:515] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.697803 8544 ts_tablet_manager.cc:1434] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20251028 03:00:03.697994 8553 leader_election.cc:290] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:03.698093 8544 tablet_bootstrap.cc:492] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.698590 8416 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a38ef10c145a4d00a26265a9dee5c744" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:03.698710 8416 raft_consensus.cc:3060] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.699424 8544 tablet_bootstrap.cc:654] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.699682 8416 raft_consensus.cc:2468] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:03.699986 8220 leader_election.cc:304] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:03.700158 8553 raft_consensus.cc:2804] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:03.700225 8553 raft_consensus.cc:697] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:03.700340 8553 consensus_queue.cc:237] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.701401 8545 tablet_bootstrap.cc:492] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.701489 8545 ts_tablet_manager.cc:1403] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20251028 03:00:03.701653 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a38ef10c145a4d00a26265a9dee5c744" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:03.701913 8545 raft_consensus.cc:359] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.702003 8545 raft_consensus.cc:385] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.702030 8545 raft_consensus.cc:740] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
W20251028 03:00:03.702033 8218 leader_election.cc:343] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.702134 8545 consensus_queue.cc:260] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.702677 8545 ts_tablet_manager.cc:1434] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.702857 8545 tablet_bootstrap.cc:492] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.703742 8545 tablet_bootstrap.cc:654] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.705685 8544 tablet_bootstrap.cc:492] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.705785 8544 ts_tablet_manager.cc:1403] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20251028 03:00:03.706251 8544 raft_consensus.cc:359] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.706338 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a38ef10c145a4d00a26265a9dee5c744" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:03.706350 8544 raft_consensus.cc:385] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.706544 8544 raft_consensus.cc:740] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
W20251028 03:00:03.706669 8218 leader_election.cc:343] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.706823 8544 consensus_queue.cc:260] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.706979 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet fb59e5f7bb57488ba2f9e4b43a118bf8 with delete_type TABLET_DATA_DELETED (replaced by tablet b53ba38b16c84b199490ce8b93316b85) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.707422 8544 ts_tablet_manager.cc:1434] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20251028 03:00:03.707440 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet fb59e5f7bb57488ba2f9e4b43a118bf8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fb59e5f7bb57488ba2f9e4b43a118bf8 already in progress: creating tablet
I20251028 03:00:03.707695 8544 tablet_bootstrap.cc:492] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.708704 8544 tablet_bootstrap.cc:654] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.709282 8545 tablet_bootstrap.cc:492] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.709383 8545 ts_tablet_manager.cc:1403] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20251028 03:00:03.709901 8545 raft_consensus.cc:359] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.710023 8545 raft_consensus.cc:385] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.710064 8545 raft_consensus.cc:740] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.710184 8545 consensus_queue.cc:260] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.710467 8545 ts_tablet_manager.cc:1434] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251028 03:00:03.710687 8545 tablet_bootstrap.cc:492] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.712285 8545 tablet_bootstrap.cc:654] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.721071 8544 tablet_bootstrap.cc:492] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.721196 8544 ts_tablet_manager.cc:1403] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.002s
I20251028 03:00:03.721768 8544 raft_consensus.cc:359] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.721877 8544 raft_consensus.cc:385] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.721905 8544 raft_consensus.cc:740] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.722035 8544 consensus_queue.cc:260] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.722316 8544 ts_tablet_manager.cc:1434] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:03.722781 8544 tablet_bootstrap.cc:492] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.722832 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_0bd1a82a268f4a798071fba7b155d35c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.723263 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0bd1a82a268f4a798071fba7b155d35c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.723863 8544 tablet_bootstrap.cc:654] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.724129 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_c725c121a0924531afb9b464c627b680"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.724453 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c725c121a0924531afb9b464c627b680 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.725278 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3dae09d02c134e68bf3159a5dc3d3053" } new_table_name: "default.Table_bc39558d01c04a3a93232e69380bf987" modify_external_catalogs: true
I20251028 03:00:03.725724 8544 tablet_bootstrap.cc:492] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.725962 8544 ts_tablet_manager.cc:1403] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251028 03:00:03.725293 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_974b645c67e142348f96878a0bc3132a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.726205 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_974b645c67e142348f96878a0bc3132a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.726387 8544 raft_consensus.cc:359] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.726475 8544 raft_consensus.cc:385] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.726501 8544 raft_consensus.cc:740] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.726599 8544 consensus_queue.cc:260] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.726871 8544 ts_tablet_manager.cc:1434] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:03.728114 8544 tablet_bootstrap.cc:492] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.729277 8544 tablet_bootstrap.cc:654] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.736075 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet a38ef10c145a4d00a26265a9dee5c744 (table default.Table_8d72f9b8a87847d0a40f66c2bcdba651 [id=ba8b6a629ebc489abe7043890455765a]). Expected version 1 got 0
I20251028 03:00:03.736402 7969 catalog_manager.cc:5649] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:03.737373 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_dc630fed688f43898c0beceb9e1c6b9a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:03.737550 8387 tablet_service.cc:1505] Processing CreateTablet for tablet b1eaae3315314bbc9b896f6c13e34be2 (DEFAULT_TABLE table=default.Table_4a26d5c6c75d402c994cc1481cd8c6a4 [id=a0770065e2cf4dfa83a543a53cc166f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.737643 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet e45fe8888fbc415ea637b363dccdd06b
W20251028 03:00:03.737768 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc630fed688f43898c0beceb9e1c6b9a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.737872 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1eaae3315314bbc9b896f6c13e34be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.738399 8265 tablet_service.cc:1505] Processing CreateTablet for tablet b1eaae3315314bbc9b896f6c13e34be2 (DEFAULT_TABLE table=default.Table_4a26d5c6c75d402c994cc1481cd8c6a4 [id=a0770065e2cf4dfa83a543a53cc166f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.738610 7963 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet e3b33c4d189247a89878b6a8e1fcbac0 (table default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]). Expected version 1 got 0
I20251028 03:00:03.738672 8256 tablet_service.cc:1505] Processing CreateTablet for tablet df45f016317b43d5a3219fd2e481756a (DEFAULT_TABLE table=default.Table_8a28a3e233bb4011b36e95509d454cc6 [id=40afefe7bc06446c98bb48eb0d7eb508]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.738726 7963 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 583d6226a5f44d47a5ea4788c2b0d5ce (table default.Table_1e52c58d3e4042139159ba0e312a4089 [id=f5dcccbdb3e44a00a0169cff89823b0e]). Expected version 1 got 0
I20251028 03:00:03.738821 7963 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 8665d2e933fa423d81a1b32f2ce8629b (table default.Table_139255bc4fb74fae8fb86563b54826bd [id=dc3721126b09466ab5efdb8bd4b46d89]). Expected version 1 got 0
I20251028 03:00:03.738919 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 4cb6e4de3cb7433fb8edec496817b593 (DEFAULT_TABLE table=default.Table_c2d77e5496bd4e578f2a583d600401cb [id=08220e1716b947d1a090305531bde065]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.739184 8261 tablet_service.cc:1505] Processing CreateTablet for tablet 2a6a8421836e412bbf11b74f7afa4f75 (DEFAULT_TABLE table=default.Table_94bf2b7d93fa48b28993b26e1abdc5e6 [id=9e96e81b5e644c2285522ab15211069f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.739500 8386 tablet_service.cc:1505] Processing CreateTablet for tablet df45f016317b43d5a3219fd2e481756a (DEFAULT_TABLE table=default.Table_8a28a3e233bb4011b36e95509d454cc6 [id=40afefe7bc06446c98bb48eb0d7eb508]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.739756 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df45f016317b43d5a3219fd2e481756a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.740731 8121 tablet_service.cc:1505] Processing CreateTablet for tablet b1eaae3315314bbc9b896f6c13e34be2 (DEFAULT_TABLE table=default.Table_4a26d5c6c75d402c994cc1481cd8c6a4 [id=a0770065e2cf4dfa83a543a53cc166f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.740792 8111 tablet_service.cc:1505] Processing CreateTablet for tablet df45f016317b43d5a3219fd2e481756a (DEFAULT_TABLE table=default.Table_8a28a3e233bb4011b36e95509d454cc6 [id=40afefe7bc06446c98bb48eb0d7eb508]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.741046 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df45f016317b43d5a3219fd2e481756a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.741080 8122 tablet_service.cc:1505] Processing CreateTablet for tablet 2a6a8421836e412bbf11b74f7afa4f75 (DEFAULT_TABLE table=default.Table_94bf2b7d93fa48b28993b26e1abdc5e6 [id=9e96e81b5e644c2285522ab15211069f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.741274 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a6a8421836e412bbf11b74f7afa4f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.741397 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 4cb6e4de3cb7433fb8edec496817b593 (DEFAULT_TABLE table=default.Table_c2d77e5496bd4e578f2a583d600401cb [id=08220e1716b947d1a090305531bde065]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.741598 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb6e4de3cb7433fb8edec496817b593. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.741933 8392 tablet_service.cc:1505] Processing CreateTablet for tablet 4cb6e4de3cb7433fb8edec496817b593 (DEFAULT_TABLE table=default.Table_c2d77e5496bd4e578f2a583d600401cb [id=08220e1716b947d1a090305531bde065]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.742141 8392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb6e4de3cb7433fb8edec496817b593. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.743708 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet e45fe8888fbc415ea637b363dccdd06b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.743891 8550 tablet_replica.cc:333] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:03.744043 8550 raft_consensus.cc:2243] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.744179 8550 raft_consensus.cc:2272] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.744736 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1eaae3315314bbc9b896f6c13e34be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.744822 8550 ts_tablet_manager.cc:1916] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.745437 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet e45fe8888fbc415ea637b363dccdd06b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.745561 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 46ae536d269d4cbdbcee44a26d25bd67 with delete_type TABLET_DATA_DELETED (replaced by tablet ada2c7c883014aa597ad8dbe14bf51b3) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.745649 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet 3435803c6167447a8b3afba6c0f3511b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.745759 8551 tablet_replica.cc:333] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.745915 8551 raft_consensus.cc:2243] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:03.746122 8551 raft_consensus.cc:2272] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.746429 7963 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 4f90e3c3524e4498aa6c4bfc5304d44b (table default.Table_42f89d7fccbf4c738293e9796be235a0 [id=6f3e689a22ae4160ae7537d955c06a90]). Expected version 1 got 0
I20251028 03:00:03.747112 8550 ts_tablet_manager.cc:1929] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:03.747213 8550 log.cc:1199] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/e45fe8888fbc415ea637b363dccdd06b
I20251028 03:00:03.747485 8550 ts_tablet_manager.cc:1950] T e45fe8888fbc415ea637b363dccdd06b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:03.748068 8551 ts_tablet_manager.cc:1916] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.749150 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet e45fe8888fbc415ea637b363dccdd06b (table default.Table_1562d90171b44a268e4777c51b3c8483 [id=7dc1def3a92c46cd8995a3b46c4e2231]) successfully deleted
I20251028 03:00:03.749488 10253 raft_consensus.cc:493] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.749573 10253 raft_consensus.cc:515] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.749992 10253 leader_election.cc:290] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.750128 8551 ts_tablet_manager.cc:1929] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:03.750200 8551 log.cc:1199] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/e45fe8888fbc415ea637b363dccdd06b
I20251028 03:00:03.750437 8551 ts_tablet_manager.cc:1950] T e45fe8888fbc415ea637b363dccdd06b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.751058 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e369dbabc2c04ad5ab17bbd4225127c1" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.751372 8396 tablet_service.cc:1552] Processing DeleteTablet for tablet bedb4fd3b46f4453b194cfaea0c3c701 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.751503 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 8e202e53053248a7a9cee6a217999489 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.751981 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 8720ea8c68904d57a3fc5c0e94d338ca with delete_type TABLET_DATA_DELETED (replaced by tablet 30232298155542ecb41f5a1b19b16ffb) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:03.752436 8083 leader_election.cc:343] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.752647 8257 tablet_service.cc:1552] Processing DeleteTablet for tablet e45fe8888fbc415ea637b363dccdd06b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:03.752889 8083 leader_election.cc:343] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: e369dbabc2c04ad5ab17bbd4225127c1
I20251028 03:00:03.752974 8083 leader_election.cc:304] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:03.753244 10253 raft_consensus.cc:2749] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.753579 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 2a6a8421836e412bbf11b74f7afa4f75 (DEFAULT_TABLE table=default.Table_94bf2b7d93fa48b28993b26e1abdc5e6 [id=9e96e81b5e644c2285522ab15211069f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:03.753852 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a6a8421836e412bbf11b74f7afa4f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.756793 8551 tablet_replica.cc:333] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.757035 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet e45fe8888fbc415ea637b363dccdd06b (table default.Table_1562d90171b44a268e4777c51b3c8483 [id=7dc1def3a92c46cd8995a3b46c4e2231]) successfully deleted
I20251028 03:00:03.757323 8551 raft_consensus.cc:2243] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.757493 8551 raft_consensus.cc:2272] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.758040 8551 ts_tablet_manager.cc:1916] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.758055 7963 catalog_manager.cc:5649] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 03:00:03.758292 7963 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet a38ef10c145a4d00a26265a9dee5c744 (table default.Table_8d72f9b8a87847d0a40f66c2bcdba651 [id=ba8b6a629ebc489abe7043890455765a]). Expected version 1 got 0
I20251028 03:00:03.760288 8551 ts_tablet_manager.cc:1929] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.760459 8551 log.cc:1199] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/46ae536d269d4cbdbcee44a26d25bd67
I20251028 03:00:03.760742 8551 ts_tablet_manager.cc:1950] T 46ae536d269d4cbdbcee44a26d25bd67 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.761426 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 46ae536d269d4cbdbcee44a26d25bd67 (table default.Table_0325897e2eb5414685dd8be9e6d2fd2e [id=b9c19c3aa17e4059b3f58533143edff6]) successfully deleted
I20251028 03:00:03.761705 8551 tablet_replica.cc:333] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.761898 10242 raft_consensus.cc:493] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.761919 8551 raft_consensus.cc:2243] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.762019 10242 raft_consensus.cc:515] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.762447 10242 leader_election.cc:290] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.762814 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bedb4fd3b46f4453b194cfaea0c3c701" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:03.762889 8551 raft_consensus.cc:2272] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.762943 8148 raft_consensus.cc:2468] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:03.763242 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bedb4fd3b46f4453b194cfaea0c3c701" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.763547 8354 leader_election.cc:304] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:03.764575 10242 raft_consensus.cc:2804] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:03.764626 10242 raft_consensus.cc:493] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:03.764660 10242 raft_consensus.cc:3060] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:03.765524 8356 leader_election.cc:343] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.766631 10242 raft_consensus.cc:515] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.766983 10242 leader_election.cc:290] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.767704 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bedb4fd3b46f4453b194cfaea0c3c701" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:03.767841 8148 raft_consensus.cc:3060] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.768188 10242 raft_consensus.cc:493] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.768263 10242 raft_consensus.cc:515] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.768587 10242 leader_election.cc:290] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.768837 8148 raft_consensus.cc:2468] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:03.770825 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bedb4fd3b46f4453b194cfaea0c3c701" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:03.771601 8354 leader_election.cc:304] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:03.771790 9190 raft_consensus.cc:2804] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:03.771886 9190 raft_consensus.cc:697] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
W20251028 03:00:03.772017 8356 leader_election.cc:343] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.772033 9190 consensus_queue.cc:237] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.772653 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8e202e53053248a7a9cee6a217999489" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:03.774796 8354 leader_election.cc:343] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.776576 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1eaae3315314bbc9b896f6c13e34be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.776649 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8e202e53053248a7a9cee6a217999489" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.778890 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df45f016317b43d5a3219fd2e481756a. 1 dirs total, 0 dirs full, 0 dirs failed
W20251028 03:00:03.779757 8356 leader_election.cc:343] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.779855 8356 leader_election.cc:304] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.780081 10245 raft_consensus.cc:493] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.780161 10245 raft_consensus.cc:515] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.780514 10245 leader_election.cc:290] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.780592 10245 raft_consensus.cc:2749] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.781339 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb6e4de3cb7433fb8edec496817b593. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.782027 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "583d6226a5f44d47a5ea4788c2b0d5ce" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:03.782725 8356 leader_election.cc:343] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.783419 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "583d6226a5f44d47a5ea4788c2b0d5ce" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:03.783598 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a6a8421836e412bbf11b74f7afa4f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:03.783809 8552 tablet_replica.cc:333] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 03:00:03.783799 8354 leader_election.cc:343] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.783866 8354 leader_election.cc:304] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.783946 8552 raft_consensus.cc:2243] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.784061 8552 raft_consensus.cc:2272] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.784058 10242 raft_consensus.cc:2749] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.784577 8552 ts_tablet_manager.cc:1916] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.786321 8551 ts_tablet_manager.cc:1916] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.786813 8552 ts_tablet_manager.cc:1929] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.786875 8552 log.cc:1199] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/e45fe8888fbc415ea637b363dccdd06b
I20251028 03:00:03.787103 8552 ts_tablet_manager.cc:1950] T e45fe8888fbc415ea637b363dccdd06b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.788414 8544 tablet_bootstrap.cc:492] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.788508 8544 ts_tablet_manager.cc:1403] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.060s user 0.001s sys 0.001s
I20251028 03:00:03.788944 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet e45fe8888fbc415ea637b363dccdd06b (table default.Table_1562d90171b44a268e4777c51b3c8483 [id=7dc1def3a92c46cd8995a3b46c4e2231]) successfully deleted
I20251028 03:00:03.789059 8544 raft_consensus.cc:359] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.789170 8544 raft_consensus.cc:385] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.789203 8544 raft_consensus.cc:740] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.789319 8544 consensus_queue.cc:260] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.789618 8544 ts_tablet_manager.cc:1434] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.789878 8544 tablet_bootstrap.cc:492] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.789909 8551 ts_tablet_manager.cc:1929] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.789969 8551 log.cc:1199] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3435803c6167447a8b3afba6c0f3511b
I20251028 03:00:03.790180 8551 ts_tablet_manager.cc:1950] T 3435803c6167447a8b3afba6c0f3511b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.790732 8551 tablet_replica.cc:333] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.790859 8551 raft_consensus.cc:2243] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:03.790982 8544 tablet_bootstrap.cc:654] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.791013 8551 pending_rounds.cc:70] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 03:00:03.791051 8551 pending_rounds.cc:77] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597173852184576 op_type: NO_OP noop_request { }
I20251028 03:00:03.791107 8551 raft_consensus.cc:2889] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:03.791168 8551 raft_consensus.cc:2272] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.794601 9190 raft_consensus.cc:493] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.794687 9190 raft_consensus.cc:515] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.795017 9190 leader_election.cc:290] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.795527 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3435803c6167447a8b3afba6c0f3511b (table default.Table_e40f83dab24e4e83b634a88cfe208817 [id=b987718414e246efb3353a3c8bd35f2f]) successfully deleted
I20251028 03:00:03.795799 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8720ea8c68904d57a3fc5c0e94d338ca" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.796190 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8720ea8c68904d57a3fc5c0e94d338ca" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:03.796510 8354 leader_election.cc:343] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:03.796594 8356 leader_election.cc:343] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.796636 8354 leader_election.cc:304] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.796806 10242 raft_consensus.cc:2749] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.798283 7962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_4d022cc0789d48a485d60399613331f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.798750 7962 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d022cc0789d48a485d60399613331f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.799610 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_98c5f51b251e48de9edb59891dbb28d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.799973 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98c5f51b251e48de9edb59891dbb28d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.800688 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_74c1ac81e5664350bfef1e9e0161543a" } new_table_name: "default.Table_a2307d8748944456aa511a1130fb949a" modify_external_catalogs: true
W20251028 03:00:03.802727 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e9b5228d4131446f8460917e961b8f15 consensus state has no leader
I20251028 03:00:03.804180 8544 tablet_bootstrap.cc:492] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.804272 8544 ts_tablet_manager.cc:1403] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.002s
I20251028 03:00:03.804242 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bfc47b32a50e48e481faae95c5e02679" } new_table_name: "default.Table_5948263bd7344940b51488668cfaa440" modify_external_catalogs: true
I20251028 03:00:03.804744 8544 raft_consensus.cc:359] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.804844 8544 raft_consensus.cc:385] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.804875 8544 raft_consensus.cc:740] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.804986 8544 consensus_queue.cc:260] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.805104 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 6a0ea69a0c3d45c4803c23787eae09ec from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:03.805301 8544 ts_tablet_manager.cc:1434] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:03.805543 8544 tablet_bootstrap.cc:492] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.806634 8544 tablet_bootstrap.cc:654] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.806735 9595 raft_consensus.cc:493] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.806838 9595 raft_consensus.cc:515] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.807212 9595 leader_election.cc:290] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:03.808008 8220 leader_election.cc:343] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 28b81f0000c4407096feff773c29579d
I20251028 03:00:03.808207 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "28b81f0000c4407096feff773c29579d" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:03.808547 8218 leader_election.cc:343] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.808610 8218 leader_election.cc:304] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.808835 9595 raft_consensus.cc:2749] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251028 03:00:03.809504 10414 consensus_queue.cc:1175] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d7e4cf694f71438298dd32a346abcf75" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20251028 03:00:03.810050 8551 ts_tablet_manager.cc:1916] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.812516 8551 ts_tablet_manager.cc:1929] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:03.812638 8551 log.cc:1199] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/bedb4fd3b46f4453b194cfaea0c3c701
I20251028 03:00:03.812675 8546 tablet_bootstrap.cc:492] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.812798 8546 ts_tablet_manager.cc:1403] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.134s user 0.002s sys 0.000s
I20251028 03:00:03.812908 8551 ts_tablet_manager.cc:1950] T bedb4fd3b46f4453b194cfaea0c3c701 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.813328 8546 raft_consensus.cc:359] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.813427 8546 raft_consensus.cc:385] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.813453 8546 raft_consensus.cc:740] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.813561 8551 tablet_replica.cc:333] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.813582 8546 consensus_queue.cc:260] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.813642 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet bedb4fd3b46f4453b194cfaea0c3c701 (table default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]) successfully deleted
I20251028 03:00:03.813711 8551 raft_consensus.cc:2243] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.813802 8551 raft_consensus.cc:2272] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.813884 8546 ts_tablet_manager.cc:1434] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.814080 8546 tablet_bootstrap.cc:492] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.814292 8551 ts_tablet_manager.cc:1916] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.815047 8546 tablet_bootstrap.cc:654] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.816583 8544 tablet_bootstrap.cc:492] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.816673 8544 ts_tablet_manager.cc:1403] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20251028 03:00:03.817118 8544 raft_consensus.cc:359] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.817210 8544 raft_consensus.cc:385] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.817236 8544 raft_consensus.cc:740] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.817369 8544 consensus_queue.cc:260] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.818886 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4a57e122bbb94a9fac8c8254ab73a21f
I20251028 03:00:03.818962 8544 ts_tablet_manager.cc:1434] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:03.819147 8544 tablet_bootstrap.cc:492] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.819268 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 4a57e122bbb94a9fac8c8254ab73a21f deleted and replaced by tablet edf3884d16ec4706b75051f72d96a3b2
I20251028 03:00:03.820749 8544 tablet_bootstrap.cc:654] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.821242 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a57e122bbb94a9fac8c8254ab73a21f with delete_type TABLET_DATA_DELETED (replaced by tablet edf3884d16ec4706b75051f72d96a3b2) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.821413 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a57e122bbb94a9fac8c8254ab73a21f with delete_type TABLET_DATA_DELETED (replaced by tablet edf3884d16ec4706b75051f72d96a3b2) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.821429 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a57e122bbb94a9fac8c8254ab73a21f with delete_type TABLET_DATA_DELETED (replaced by tablet edf3884d16ec4706b75051f72d96a3b2) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.822499 8551 ts_tablet_manager.cc:1929] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.822564 8551 log.cc:1199] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8e202e53053248a7a9cee6a217999489
I20251028 03:00:03.822782 8551 ts_tablet_manager.cc:1950] T 8e202e53053248a7a9cee6a217999489 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 03:00:03.823354 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4a57e122bbb94a9fac8c8254ab73a21f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a57e122bbb94a9fac8c8254ab73a21f already in progress: creating tablet
W20251028 03:00:03.823798 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 4a57e122bbb94a9fac8c8254ab73a21f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a57e122bbb94a9fac8c8254ab73a21f already in progress: creating tablet
I20251028 03:00:03.828416 9595 raft_consensus.cc:493] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.828523 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8e202e53053248a7a9cee6a217999489 (table default.Table_01d864857be34675882e1be7ba5a9a7c [id=32dae7dc982547979850fa8ad8a80de9]) successfully deleted
I20251028 03:00:03.828533 9595 raft_consensus.cc:515] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.828928 9595 leader_election.cc:290] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:03.829663 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e9ad4ae17665483584ce2d638d1639fb" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:03.829794 8220 leader_election.cc:343] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: e9ad4ae17665483584ce2d638d1639fb
W20251028 03:00:03.830029 8218 leader_election.cc:343] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.830097 8218 leader_election.cc:304] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.830104 8551 tablet_replica.cc:333] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:03.830245 8551 raft_consensus.cc:2243] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.830291 9595 raft_consensus.cc:2749] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.830340 8551 raft_consensus.cc:2272] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.830902 8551 ts_tablet_manager.cc:1916] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.831736 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_25c42d82559d47c6a78d15d5ffb4b4f7" } modify_external_catalogs: true
I20251028 03:00:03.831873 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_25c42d82559d47c6a78d15d5ffb4b4f7" } modify_external_catalogs: true
I20251028 03:00:03.833384 8545 tablet_bootstrap.cc:492] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.833485 8545 ts_tablet_manager.cc:1403] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.123s user 0.002s sys 0.000s
I20251028 03:00:03.834060 8545 raft_consensus.cc:359] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.834170 8545 raft_consensus.cc:385] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.834203 8545 raft_consensus.cc:740] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.834323 8545 consensus_queue.cc:260] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.834595 8545 ts_tablet_manager.cc:1434] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.834810 8545 tablet_bootstrap.cc:492] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.839497 8545 tablet_bootstrap.cc:654] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.842377 8544 tablet_bootstrap.cc:492] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.842772 8544 ts_tablet_manager.cc:1403] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.001s
I20251028 03:00:03.843199 10253 raft_consensus.cc:493] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.843325 10253 raft_consensus.cc:515] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.843389 8544 raft_consensus.cc:359] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.843488 8544 raft_consensus.cc:385] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.843516 8544 raft_consensus.cc:740] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.843631 8544 consensus_queue.cc:260] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.843722 10253 leader_election.cc:290] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.843986 8551 ts_tablet_manager.cc:1929] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.844069 8551 log.cc:1199] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8720ea8c68904d57a3fc5c0e94d338ca
I20251028 03:00:03.844192 8544 ts_tablet_manager.cc:1434] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.844271 8551 ts_tablet_manager.cc:1950] T 8720ea8c68904d57a3fc5c0e94d338ca P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.844937 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8720ea8c68904d57a3fc5c0e94d338ca (table default.Table_79ced5936022425c91473064b87986d3 [id=375c6744f94644199d8d6f08f21ea021]) successfully deleted
I20251028 03:00:03.844944 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee2fd98744ff44028f36bb55e6eba51e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:03.845073 8419 raft_consensus.cc:2468] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:03.845132 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee2fd98744ff44028f36bb55e6eba51e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.845587 8083 leader_election.cc:304] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:03.845804 10253 raft_consensus.cc:2804] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:03.845852 10253 raft_consensus.cc:493] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:03.845885 10253 raft_consensus.cc:3060] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.846920 10253 raft_consensus.cc:515] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.847285 10253 leader_election.cc:290] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:03.847268 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4a57e122bbb94a9fac8c8254ab73a21f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a57e122bbb94a9fac8c8254ab73a21f already in progress: creating tablet
I20251028 03:00:03.847859 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee2fd98744ff44028f36bb55e6eba51e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:03.847988 8419 raft_consensus.cc:3060] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:03.848085 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee2fd98744ff44028f36bb55e6eba51e" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:03.848392 8083 leader_election.cc:343] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:03.848727 8083 leader_election.cc:343] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.849093 8419 raft_consensus.cc:2468] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:03.850137 8083 leader_election.cc:304] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:03.850384 10253 raft_consensus.cc:2804] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:03.850456 10253 raft_consensus.cc:697] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:03.850572 10253 consensus_queue.cc:237] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.850759 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 5a8e8f1a333240c2b40a0e59c3a04172
I20251028 03:00:03.851586 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 5a8e8f1a333240c2b40a0e59c3a04172 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.851596 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 5a8e8f1a333240c2b40a0e59c3a04172 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
W20251028 03:00:03.851989 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 5a8e8f1a333240c2b40a0e59c3a04172 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a8e8f1a333240c2b40a0e59c3a04172 already in progress: creating tablet
W20251028 03:00:03.852335 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 5a8e8f1a333240c2b40a0e59c3a04172 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a8e8f1a333240c2b40a0e59c3a04172 already in progress: creating tablet
I20251028 03:00:03.852375 8544 tablet_bootstrap.cc:492] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.853607 8544 tablet_bootstrap.cc:654] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.855827 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 5a8e8f1a333240c2b40a0e59c3a04172 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:03.857434 7964 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_932eaf48b5614bbfb5d581cfd42f16d2" } modify_external_catalogs: true
I20251028 03:00:03.857556 7964 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_932eaf48b5614bbfb5d581cfd42f16d2" } modify_external_catalogs: true
I20251028 03:00:03.857564 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b with delete_type TABLET_DATA_DELETED (replaced by tablet f898e9072d7d4697aee6a981cf6188b4) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.857735 8551 tablet_replica.cc:333] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
W20251028 03:00:03.857767 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 5a8e8f1a333240c2b40a0e59c3a04172 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5a8e8f1a333240c2b40a0e59c3a04172 already in progress: creating tablet
I20251028 03:00:03.857878 8551 raft_consensus.cc:2243] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.858035 8551 raft_consensus.cc:2272] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.859126 8551 ts_tablet_manager.cc:1916] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.862033 8551 ts_tablet_manager.cc:1929] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.862115 8551 log.cc:1199] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/2fec0adb79d04469b90c1ea2a5e9ba5b
I20251028 03:00:03.862329 8551 ts_tablet_manager.cc:1950] T 2fec0adb79d04469b90c1ea2a5e9ba5b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:03.863214 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 2fec0adb79d04469b90c1ea2a5e9ba5b (table default.Table_cc716ae9a4cd4b939b2e461342f03331 [id=7375e54ec4fc4747bf8e8db863ac69ff]) successfully deleted
I20251028 03:00:03.864219 8544 tablet_bootstrap.cc:492] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.864315 8544 ts_tablet_manager.cc:1403] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20251028 03:00:03.864763 8544 raft_consensus.cc:359] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.864885 8544 raft_consensus.cc:385] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.864923 8544 raft_consensus.cc:740] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.865067 8544 consensus_queue.cc:260] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.865375 10187 tablet.cc:1722] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:03.866864 8544 ts_tablet_manager.cc:1434] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:03.867065 8544 tablet_bootstrap.cc:492] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.868127 8544 tablet_bootstrap.cc:654] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.873179 10335 raft_consensus.cc:493] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.873311 10335 raft_consensus.cc:515] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.873688 10335 leader_election.cc:290] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.874468 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e4d980617994db4a188a98bb31eff92" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:03.874487 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e4d980617994db4a188a98bb31eff92" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:03.874909 8083 leader_election.cc:343] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:03.875216 8083 leader_election.cc:343] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.875278 8083 leader_election.cc:304] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:03.875540 10335 raft_consensus.cc:2749] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.879732 10422 tablet.cc:1722] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:03.882743 10335 raft_consensus.cc:493] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.882869 10335 raft_consensus.cc:515] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.883231 10335 leader_election.cc:290] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:03.883852 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b0c593a3b96434db9ac42d260fc6997" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:03.884200 8083 leader_election.cc:343] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.884856 8546 tablet_bootstrap.cc:492] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.884950 8546 ts_tablet_manager.cc:1403] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.071s user 0.000s sys 0.002s
I20251028 03:00:03.885078 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b0c593a3b96434db9ac42d260fc6997" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:03.885433 8083 leader_election.cc:343] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.885500 8083 leader_election.cc:304] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:03.885502 8546 raft_consensus.cc:359] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.885607 8546 raft_consensus.cc:385] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.885640 8546 raft_consensus.cc:740] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.885723 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b with delete_type TABLET_DATA_DELETED (replaced by tablet f898e9072d7d4697aee6a981cf6188b4) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:03.885776 8546 consensus_queue.cc:260] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.885862 8545 tablet_bootstrap.cc:492] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.885888 10335 raft_consensus.cc:2749] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.885941 8545 ts_tablet_manager.cc:1403] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.051s user 0.000s sys 0.003s
I20251028 03:00:03.886089 8546 ts_tablet_manager.cc:1434] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:03.886266 8546 tablet_bootstrap.cc:492] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.886411 8545 raft_consensus.cc:359] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.886502 8545 raft_consensus.cc:385] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.886531 8545 raft_consensus.cc:740] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.886644 8545 consensus_queue.cc:260] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.886950 8545 ts_tablet_manager.cc:1434] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:03.887163 8545 tablet_bootstrap.cc:492] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.888170 8545 tablet_bootstrap.cc:654] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.888425 8544 tablet_bootstrap.cc:492] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.888507 8544 ts_tablet_manager.cc:1403] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20251028 03:00:03.888582 8546 tablet_bootstrap.cc:654] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.888971 8544 raft_consensus.cc:359] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.889071 8544 raft_consensus.cc:385] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.889101 8544 raft_consensus.cc:740] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.889243 8544 consensus_queue.cc:260] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.893525 8544 ts_tablet_manager.cc:1434] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 03:00:03.893816 8544 tablet_bootstrap.cc:492] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.895080 8544 tablet_bootstrap.cc:654] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:03.897850 7955 catalog_manager.cc:4952] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b because the tablet was not found. No further retry: Not found: Tablet not found: 2fec0adb79d04469b90c1ea2a5e9ba5b
I20251028 03:00:03.899472 10423 tablet.cc:1722] T 338cf9d02142436f97ccd44313cc8199 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:03.910462 8544 tablet_bootstrap.cc:492] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.910569 8544 ts_tablet_manager.cc:1403] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20251028 03:00:03.910656 8546 tablet_bootstrap.cc:492] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:03.910759 8546 ts_tablet_manager.cc:1403] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20251028 03:00:03.911022 8544 raft_consensus.cc:359] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.911123 8544 raft_consensus.cc:385] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.911150 8544 raft_consensus.cc:740] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.911257 8544 consensus_queue.cc:260] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.911473 8546 raft_consensus.cc:359] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.911545 8544 ts_tablet_manager.cc:1434] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.911581 8546 raft_consensus.cc:385] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.911610 8546 raft_consensus.cc:740] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.911710 8544 tablet_bootstrap.cc:492] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.911736 8546 consensus_queue.cc:260] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.912050 8546 ts_tablet_manager.cc:1434] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.912542 8545 tablet_bootstrap.cc:492] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.912628 8545 ts_tablet_manager.cc:1403] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.001s
I20251028 03:00:03.912676 8544 tablet_bootstrap.cc:654] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.913091 8545 raft_consensus.cc:359] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.913192 8545 raft_consensus.cc:385] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.913223 8545 raft_consensus.cc:740] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.913350 8545 consensus_queue.cc:260] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.913667 8545 ts_tablet_manager.cc:1434] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.913703 8546 tablet_bootstrap.cc:492] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:03.913865 8545 tablet_bootstrap.cc:492] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.915225 8546 tablet_bootstrap.cc:654] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.917071 10335 raft_consensus.cc:493] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.917161 10335 raft_consensus.cc:515] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.917588 10335 leader_election.cc:290] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:03.918076 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a4bcdc8a0c554b7cae81d7993e4f2861" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:03.918427 8083 leader_election.cc:343] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:03.918747 8083 leader_election.cc:343] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: a4bcdc8a0c554b7cae81d7993e4f2861
I20251028 03:00:03.918817 8083 leader_election.cc:304] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:03.919198 10335 raft_consensus.cc:2749] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.920609 8545 tablet_bootstrap.cc:654] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.930583 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet d60fc654f6af44b290d46235082a4113 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d42a5987fa540ce86afdbbe1922896e) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.930809 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet 28b81f0000c4407096feff773c29579d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.930868 8552 tablet_replica.cc:333] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.930912 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ad4ae17665483584ce2d638d1639fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.930997 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet 46ae536d269d4cbdbcee44a26d25bd67 with delete_type TABLET_DATA_DELETED (replaced by tablet ada2c7c883014aa597ad8dbe14bf51b3) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:03.931061 8552 raft_consensus.cc:2243] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.931170 8552 raft_consensus.cc:2272] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.932178 8544 tablet_bootstrap.cc:492] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.932279 8544 ts_tablet_manager.cc:1403] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20251028 03:00:03.932801 8544 raft_consensus.cc:359] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.932909 8544 raft_consensus.cc:385] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.932937 8544 raft_consensus.cc:740] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.933081 8544 consensus_queue.cc:260] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.933362 8544 ts_tablet_manager.cc:1434] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.933565 8544 tablet_bootstrap.cc:492] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.934252 10242 consensus_queue.cc:1048] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:03.934639 8544 tablet_bootstrap.cc:654] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.935509 8552 ts_tablet_manager.cc:1916] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.936690 8545 tablet_bootstrap.cc:492] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.936786 8545 ts_tablet_manager.cc:1403] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20251028 03:00:03.937294 8545 raft_consensus.cc:359] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.937417 8545 raft_consensus.cc:385] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.937446 8545 raft_consensus.cc:740] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.937563 8545 consensus_queue.cc:260] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.937853 8545 ts_tablet_manager.cc:1434] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.938059 8552 ts_tablet_manager.cc:1929] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.938131 8552 log.cc:1199] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/d60fc654f6af44b290d46235082a4113
I20251028 03:00:03.938346 8552 ts_tablet_manager.cc:1950] T d60fc654f6af44b290d46235082a4113 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.939405 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet d60fc654f6af44b290d46235082a4113 (table default.Table_a387569232e54415a0dd77e906ddcee4 [id=94895dadf0e34a19a454e8755c61c785]) successfully deleted
I20251028 03:00:03.939924 8552 tablet_replica.cc:333] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.940117 8552 raft_consensus.cc:2243] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.940302 8552 raft_consensus.cc:2272] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:03.941402 8552 ts_tablet_manager.cc:1916] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.941386 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_139255bc4fb74fae8fb86563b54826bd" } modify_external_catalogs: true
I20251028 03:00:03.941509 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_139255bc4fb74fae8fb86563b54826bd" } modify_external_catalogs: true
I20251028 03:00:03.942378 8553 raft_consensus.cc:493] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.942463 8553 raft_consensus.cc:515] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.942858 8553 leader_election.cc:290] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:03.943765 8552 ts_tablet_manager.cc:1929] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.943903 8552 log.cc:1199] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/28b81f0000c4407096feff773c29579d
I20251028 03:00:03.944190 8552 ts_tablet_manager.cc:1950] T 28b81f0000c4407096feff773c29579d P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.944953 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 28b81f0000c4407096feff773c29579d (table default.Table_fa28ec70564a473fb82a04887b59f553 [id=682408eb6dfa4bc28bd0c7b9175a4c09]) successfully deleted
I20251028 03:00:03.945259 8552 tablet_replica.cc:333] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:03.945431 8552 raft_consensus.cc:2243] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:03.945583 8552 raft_consensus.cc:2272] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:03.945554 8220 leader_election.cc:343] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: fbc0337747a74e20b22aac654c2ab7cc
W20251028 03:00:03.946071 8218 leader_election.cc:343] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: fbc0337747a74e20b22aac654c2ab7cc
I20251028 03:00:03.946146 8218 leader_election.cc:304] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:03.946422 8553 raft_consensus.cc:2749] T fbc0337747a74e20b22aac654c2ab7cc P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:03.946931 8552 ts_tablet_manager.cc:1916] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:03.947993 8545 tablet_bootstrap.cc:492] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.949975 8552 ts_tablet_manager.cc:1929] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:03.950121 8552 log.cc:1199] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/e9ad4ae17665483584ce2d638d1639fb
I20251028 03:00:03.950373 8545 tablet_bootstrap.cc:654] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.950407 8552 ts_tablet_manager.cc:1950] T e9ad4ae17665483584ce2d638d1639fb P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:03.963325 10253 raft_consensus.cc:493] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:03.963454 10253 raft_consensus.cc:515] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.963828 10253 leader_election.cc:290] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:03.964690 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d5c3b02db24d4dc48260b42841844938" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
W20251028 03:00:03.965873 8083 leader_election.cc:343] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:03.965909 8544 tablet_bootstrap.cc:492] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.966099 8544 ts_tablet_manager.cc:1403] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.001s
I20251028 03:00:03.966571 8544 raft_consensus.cc:359] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.966660 8544 raft_consensus.cc:385] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.966686 8544 raft_consensus.cc:740] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.966799 8544 consensus_queue.cc:260] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:03.967048 8544 ts_tablet_manager.cc:1434] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.967221 8544 tablet_bootstrap.cc:492] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.970577 8544 tablet_bootstrap.cc:654] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.976935 8544 tablet_bootstrap.cc:492] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.977057 8544 ts_tablet_manager.cc:1403] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 03:00:03.977561 8544 raft_consensus.cc:359] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.977682 8544 raft_consensus.cc:385] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.977722 8544 raft_consensus.cc:740] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.977845 8544 consensus_queue.cc:260] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.978166 8544 ts_tablet_manager.cc:1434] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:03.978520 8544 tablet_bootstrap.cc:492] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:03.979550 8544 tablet_bootstrap.cc:654] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.980443 8545 tablet_bootstrap.cc:492] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:03.980540 8545 ts_tablet_manager.cc:1403] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.001s
I20251028 03:00:03.981024 8545 raft_consensus.cc:359] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.981129 8545 raft_consensus.cc:385] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.981160 8545 raft_consensus.cc:740] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.981422 8545 consensus_queue.cc:260] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.983258 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_1583dbc0fd384dea83db72bc0b3c8354"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.983777 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1583dbc0fd384dea83db72bc0b3c8354 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.985934 10253 consensus_queue.cc:1048] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:03.987679 8544 tablet_bootstrap.cc:492] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.987782 8544 ts_tablet_manager.cc:1403] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20251028 03:00:03.988240 8544 raft_consensus.cc:359] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.988332 8544 raft_consensus.cc:385] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.988360 8544 raft_consensus.cc:740] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.988466 8544 consensus_queue.cc:260] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.988725 8544 ts_tablet_manager.cc:1434] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:03.988906 8544 tablet_bootstrap.cc:492] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 03:00:03.990118 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 517df20e0f3245cfb1eaf1eb52d3c6a8 consensus state has no leader
I20251028 03:00:03.990320 7961 master_service.cc:954] ReplaceTablet: received request to replace tablet 1f57930d5211413cb6cf4a693f7557d1 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:03.990447 8544 tablet_bootstrap.cc:654] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.992571 8545 ts_tablet_manager.cc:1434] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20251028 03:00:03.992914 8545 tablet_bootstrap.cc:492] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:03.994037 8545 tablet_bootstrap.cc:654] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:03.996342 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_311b5d236e6248b5958556c596d0d986"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:03.996819 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_311b5d236e6248b5958556c596d0d986 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:03.998967 8544 tablet_bootstrap.cc:492] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:03.999065 8544 ts_tablet_manager.cc:1403] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20251028 03:00:03.999545 8544 raft_consensus.cc:359] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:03.999639 8544 raft_consensus.cc:385] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:03.999665 8544 raft_consensus.cc:740] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:03.999779 8544 consensus_queue.cc:260] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.000029 8544 ts_tablet_manager.cc:1434] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.000193 8544 tablet_bootstrap.cc:492] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.001201 8544 tablet_bootstrap.cc:654] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.004483 8544 tablet_bootstrap.cc:492] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.004573 8544 ts_tablet_manager.cc:1403] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
W20251028 03:00:04.004662 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99cc3e2b1eef4137bac27c629600f43b consensus state has no leader
I20251028 03:00:04.005013 8544 raft_consensus.cc:359] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.005125 8544 raft_consensus.cc:385] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.005156 8544 raft_consensus.cc:740] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.006402 8544 consensus_queue.cc:260] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.006877 8544 ts_tablet_manager.cc:1434] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:04.007103 8544 tablet_bootstrap.cc:492] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.008167 8544 tablet_bootstrap.cc:654] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:04.017014 8083 consensus_peers.cc:597] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.022500 10253 raft_consensus.cc:493] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.022629 10253 raft_consensus.cc:515] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.023022 10253 leader_election.cc:290] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:04.024061 8083 leader_election.cc:343] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 3435803c6167447a8b3afba6c0f3511b
I20251028 03:00:03.964504 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d5c3b02db24d4dc48260b42841844938" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:03.972764 8546 tablet_bootstrap.cc:492] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.023773 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3435803c6167447a8b3afba6c0f3511b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.027593 8546 ts_tablet_manager.cc:1403] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.114s user 0.002s sys 0.000s
I20251028 03:00:04.028103 8546 raft_consensus.cc:359] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.028213 8546 raft_consensus.cc:385] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.028242 8546 raft_consensus.cc:740] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.028347 8546 consensus_queue.cc:260] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.028616 8546 ts_tablet_manager.cc:1434] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.028825 8546 tablet_bootstrap.cc:492] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.029772 8546 tablet_bootstrap.cc:654] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:04.030464 8083 leader_election.cc:343] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.030546 8083 leader_election.cc:304] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:04.030689 10253 raft_consensus.cc:2749] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251028 03:00:04.030790 8083 leader_election.cc:343] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.030848 8083 leader_election.cc:304] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:04.030954 10253 raft_consensus.cc:2749] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.035167 10253 raft_consensus.cc:493] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.035279 10253 raft_consensus.cc:515] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.035676 10253 leader_election.cc:290] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:04.036391 8083 leader_election.cc:343] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 46ae536d269d4cbdbcee44a26d25bd67
I20251028 03:00:04.036535 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "46ae536d269d4cbdbcee44a26d25bd67" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.039839 8083 leader_election.cc:304] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:04.040129 10335 raft_consensus.cc:2749] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.044621 8552 tablet_replica.cc:333] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.044801 8552 raft_consensus.cc:2243] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.044941 8552 pending_rounds.cc:70] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 03:00:04.044967 8552 pending_rounds.cc:77] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597173302325248 op_type: NO_OP noop_request { }
I20251028 03:00:04.045020 8552 raft_consensus.cc:2889] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:04.045063 8552 raft_consensus.cc:2272] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.046028 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet e9ad4ae17665483584ce2d638d1639fb (table default.Table_929b382a93794515a84b4f6356fb86b2 [id=5f35ae4ce3914890a3ce64198fbdb39e]) successfully deleted
I20251028 03:00:04.046306 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3d306057aa1a4c9cb80d782c62f67d76" } new_table_name: "default.Table_32faa291266d4d11b78e09816341a9a9" modify_external_catalogs: true
I20251028 03:00:04.046586 10242 raft_consensus.cc:493] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.046689 10242 raft_consensus.cc:515] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.047094 10242 leader_election.cc:290] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.051035 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6a0ea69a0c3d45c4803c23787eae09ec
I20251028 03:00:04.051386 7965 catalog_manager.cc:6719] ReplaceTablet: tablet 6a0ea69a0c3d45c4803c23787eae09ec deleted and replaced by tablet ece8bffc9f8347e6bf87808d3a61fb7d
I20251028 03:00:04.051864 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a0ea69a0c3d45c4803c23787eae09ec with delete_type TABLET_DATA_DELETED (replaced by tablet ece8bffc9f8347e6bf87808d3a61fb7d) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.052130 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a0ea69a0c3d45c4803c23787eae09ec with delete_type TABLET_DATA_DELETED (replaced by tablet ece8bffc9f8347e6bf87808d3a61fb7d) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.055063 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b787f8f024745b297b2ebaa456cbcbf" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.055207 8273 raft_consensus.cc:2468] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:04.055547 8356 leader_election.cc:304] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:04.055742 9190 raft_consensus.cc:2804] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.055804 9190 raft_consensus.cc:493] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.055840 9190 raft_consensus.cc:3060] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.055914 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b787f8f024745b297b2ebaa456cbcbf" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.056037 8148 raft_consensus.cc:2468] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:04.056988 9190 raft_consensus.cc:515] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.057381 9190 leader_election.cc:290] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:04.058604 8356 consensus_peers.cc:597] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.059052 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b787f8f024745b297b2ebaa456cbcbf" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:04.059177 8283 raft_consensus.cc:3060] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.060314 8283 raft_consensus.cc:2468] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:04.060762 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4b787f8f024745b297b2ebaa456cbcbf" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:04.060879 8148 raft_consensus.cc:3060] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.061225 8356 leader_election.cc:304] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:04.061715 9190 raft_consensus.cc:2804] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.061825 9190 raft_consensus.cc:697] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:04.061849 8148 raft_consensus.cc:2468] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:04.061959 9190 consensus_queue.cc:237] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.062620 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 6a0ea69a0c3d45c4803c23787eae09ec with delete_type TABLET_DATA_DELETED (replaced by tablet ece8bffc9f8347e6bf87808d3a61fb7d) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:04.063475 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6a0ea69a0c3d45c4803c23787eae09ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a0ea69a0c3d45c4803c23787eae09ec already in progress: creating tablet
W20251028 03:00:04.065683 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 6a0ea69a0c3d45c4803c23787eae09ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a0ea69a0c3d45c4803c23787eae09ec already in progress: creating tablet
I20251028 03:00:04.066532 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ff79527661d5411d9933e17bf3c6675c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.067025 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff79527661d5411d9933e17bf3c6675c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:04.067353 8083 consensus_peers.cc:597] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.070531 10335 raft_consensus.cc:493] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.070636 10335 raft_consensus.cc:515] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.071039 10335 leader_election.cc:290] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.071741 10245 raft_consensus.cc:493] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.071844 10245 raft_consensus.cc:515] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.072201 10245 leader_election.cc:290] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:04.072646 8083 leader_election.cc:343] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: e9ad4ae17665483584ce2d638d1639fb
I20251028 03:00:04.072834 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f90e3c3524e4498aa6c4bfc5304d44b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.072969 8148 raft_consensus.cc:2468] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
W20251028 03:00:04.073598 8083 leader_election.cc:343] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: e9ad4ae17665483584ce2d638d1639fb
I20251028 03:00:04.073678 8083 leader_election.cc:304] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:04.073704 8354 leader_election.cc:304] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:04.073920 10253 raft_consensus.cc:2749] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.074046 9190 raft_consensus.cc:2804] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.074092 9190 raft_consensus.cc:493] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.074131 9190 raft_consensus.cc:3060] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.076089 10335 consensus_queue.cc:1048] T d0dad91f926648c781d1582733d6ae95 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.076452 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f90e3c3524e4498aa6c4bfc5304d44b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:04.077020 8356 leader_election.cc:343] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.077178 9190 raft_consensus.cc:515] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.077517 9190 leader_election.cc:290] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.082947 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f90e3c3524e4498aa6c4bfc5304d44b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:04.083094 8148 raft_consensus.cc:3060] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.084302 8148 raft_consensus.cc:2468] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:04.086046 8552 ts_tablet_manager.cc:1916] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.086386 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 8665d2e933fa423d81a1b32f2ce8629b
I20251028 03:00:04.086932 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_b487827beda64cfe9fc5cdf1daa37a4d" } new_table_name: "default.Table_e3dcc51ec84b47d2ac54d1b06f10efa5" modify_external_catalogs: true
I20251028 03:00:04.087332 7964 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet da46096a2001470ba1bd1c9660e4b0c5
I20251028 03:00:04.087673 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_9e3203e9487048d881979b269f95368f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:04.087651 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_221b934925544c1180e7a1ee9c651c8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.088063 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_221b934925544c1180e7a1ee9c651c8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.088766 10335 raft_consensus.cc:493] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.088866 10335 raft_consensus.cc:515] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.089057 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_cd1078befff84198beaa6679f94a351c" } modify_external_catalogs: true
I20251028 03:00:04.089144 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_cd1078befff84198beaa6679f94a351c" } modify_external_catalogs: true
I20251028 03:00:04.089303 10335 leader_election.cc:290] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.089610 8354 leader_election.cc:304] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:04.091064 10423 tablet.cc:1722] T 00550a40f04046d19e9112814372da6c P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251028 03:00:04.088063 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e3203e9487048d881979b269f95368f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.091315 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet 8665d2e933fa423d81a1b32f2ce8629b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.091481 8550 tablet_replica.cc:333] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.091648 8550 raft_consensus.cc:2243] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.091777 8550 raft_consensus.cc:2272] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.091948 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet da46096a2001470ba1bd1c9660e4b0c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.092427 8550 ts_tablet_manager.cc:1916] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.092808 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 8665d2e933fa423d81a1b32f2ce8629b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.092886 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet da46096a2001470ba1bd1c9660e4b0c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.093050 10245 raft_consensus.cc:2804] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.093123 10245 raft_consensus.cc:697] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:04.093250 10245 consensus_queue.cc:237] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.095007 8550 ts_tablet_manager.cc:1929] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.095110 8550 log.cc:1199] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8665d2e933fa423d81a1b32f2ce8629b
I20251028 03:00:04.095952 8550 ts_tablet_manager.cc:1950] T 8665d2e933fa423d81a1b32f2ce8629b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.097787 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a38ef10c145a4d00a26265a9dee5c744" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.097965 8420 raft_consensus.cc:2393] T a38ef10c145a4d00a26265a9dee5c744 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbd2afb4894c4e7a8520cc09f8593aa0 in current term 1: Already voted for candidate ccbdf478f18948d19de96574a685bfda in this term.
I20251028 03:00:04.098400 10335 raft_consensus.cc:493] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.098489 10335 raft_consensus.cc:515] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.098904 10335 leader_election.cc:290] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.100047 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8665d2e933fa423d81a1b32f2ce8629b (table default.Table_139255bc4fb74fae8fb86563b54826bd [id=dc3721126b09466ab5efdb8bd4b46d89]) successfully deleted
W20251028 03:00:04.100293 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet da46096a2001470ba1bd1c9660e4b0c5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da46096a2001470ba1bd1c9660e4b0c5 already in progress: creating tablet
I20251028 03:00:04.100905 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee1d686a0e4c47f6b214224047a55747" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.101038 8420 raft_consensus.cc:2468] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:04.102489 8083 leader_election.cc:304] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:04.102761 10253 raft_consensus.cc:2804] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.102819 10253 raft_consensus.cc:493] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.102860 10253 raft_consensus.cc:3060] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.104079 10253 raft_consensus.cc:515] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.104486 10253 leader_election.cc:290] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.105700 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee1d686a0e4c47f6b214224047a55747" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.105835 8420 raft_consensus.cc:3060] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.106973 8420 raft_consensus.cc:2468] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:04.107757 8083 leader_election.cc:304] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:04.107954 10335 raft_consensus.cc:2804] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.108044 10335 raft_consensus.cc:697] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:04.108180 10335 consensus_queue.cc:237] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.110172 8551 tablet_replica.cc:333] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.110358 8551 raft_consensus.cc:2243] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.110589 8551 raft_consensus.cc:2272] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.111053 8551 ts_tablet_manager.cc:1916] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.120519 8551 ts_tablet_manager.cc:1929] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.123147 8551 log.cc:1199] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/8665d2e933fa423d81a1b32f2ce8629b
I20251028 03:00:04.123939 8551 ts_tablet_manager.cc:1950] T 8665d2e933fa423d81a1b32f2ce8629b P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.124969 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 8665d2e933fa423d81a1b32f2ce8629b (table default.Table_139255bc4fb74fae8fb86563b54826bd [id=dc3721126b09466ab5efdb8bd4b46d89]) successfully deleted
W20251028 03:00:04.125487 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet da46096a2001470ba1bd1c9660e4b0c5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da46096a2001470ba1bd1c9660e4b0c5 already in progress: creating tablet
I20251028 03:00:04.126272 8545 tablet_bootstrap.cc:492] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.126366 8545 ts_tablet_manager.cc:1403] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.134s user 0.002s sys 0.001s
I20251028 03:00:04.126837 8545 raft_consensus.cc:359] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.126945 8545 raft_consensus.cc:385] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.126974 8545 raft_consensus.cc:740] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.127089 8545 consensus_queue.cc:260] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.127386 8545 ts_tablet_manager.cc:1434] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.127599 8545 tablet_bootstrap.cc:492] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.128574 8545 tablet_bootstrap.cc:654] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.090406 10187 tablet.cc:1722] T 00550a40f04046d19e9112814372da6c P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:04.090730 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet 8665d2e933fa423d81a1b32f2ce8629b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.090830 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet da46096a2001470ba1bd1c9660e4b0c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.090839 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f90e3c3524e4498aa6c4bfc5304d44b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:04.097832 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a38ef10c145a4d00a26265a9dee5c744" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.133476 10422 tablet.cc:1722] T 00550a40f04046d19e9112814372da6c P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:04.100744 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee1d686a0e4c47f6b214224047a55747" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.133633 8274 raft_consensus.cc:2468] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
W20251028 03:00:04.133661 8356 leader_election.cc:343] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.105500 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee1d686a0e4c47f6b214224047a55747" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:04.133914 8270 raft_consensus.cc:3060] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.135054 8270 raft_consensus.cc:2468] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:04.137531 8546 tablet_bootstrap.cc:492] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.137634 8546 ts_tablet_manager.cc:1403] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.109s user 0.002s sys 0.000s
I20251028 03:00:04.137678 8083 leader_election.cc:304] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:04.138128 10335 raft_consensus.cc:3060] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.138160 8546 raft_consensus.cc:359] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.138262 8546 raft_consensus.cc:385] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.138301 8546 raft_consensus.cc:740] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.138409 8546 consensus_queue.cc:260] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.138676 8546 ts_tablet_manager.cc:1434] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:04.138871 8546 tablet_bootstrap.cc:492] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.139165 10335 raft_consensus.cc:2749] T a38ef10c145a4d00a26265a9dee5c744 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.140123 10253 raft_consensus.cc:493] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.140205 10253 raft_consensus.cc:515] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.140566 8546 tablet_bootstrap.cc:654] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.140609 10253 leader_election.cc:290] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.140708 10253 raft_consensus.cc:493] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.140775 10253 raft_consensus.cc:515] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.141098 10253 leader_election.cc:290] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.141175 10253 raft_consensus.cc:493] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.141227 10253 raft_consensus.cc:515] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.141463 10253 leader_election.cc:290] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.141636 8552 ts_tablet_manager.cc:1929] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.141703 8552 log.cc:1199] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/46ae536d269d4cbdbcee44a26d25bd67
I20251028 03:00:04.141963 8552 ts_tablet_manager.cc:1950] T 46ae536d269d4cbdbcee44a26d25bd67 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.143210 8553 consensus_queue.cc:1048] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.143407 8544 tablet_bootstrap.cc:492] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.143496 8544 ts_tablet_manager.cc:1403] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.136s user 0.002s sys 0.000s
I20251028 03:00:04.144018 8544 raft_consensus.cc:359] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.144115 8544 raft_consensus.cc:385] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.144145 8544 raft_consensus.cc:740] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.144246 8544 consensus_queue.cc:260] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.144533 8544 ts_tablet_manager.cc:1434] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.144716 8544 tablet_bootstrap.cc:492] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 03:00:04.144999 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0696349affc34573b08bcb84df29c201 consensus state has no leader
I20251028 03:00:04.145196 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet 9a81841c22ae49c0a17931d90407accc from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:04.145687 8544 tablet_bootstrap.cc:654] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.146085 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 46ae536d269d4cbdbcee44a26d25bd67 (table default.Table_0325897e2eb5414685dd8be9e6d2fd2e [id=b9c19c3aa17e4059b3f58533143edff6]) successfully deleted
I20251028 03:00:04.146257 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6dd563e13e624d4bb88950b46d480876" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.146365 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d60fc654f6af44b290d46235082a4113" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.146373 8421 raft_consensus.cc:2468] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:04.148012 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6dd563e13e624d4bb88950b46d480876" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:04.148099 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 6a0ea69a0c3d45c4803c23787eae09ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a0ea69a0c3d45c4803c23787eae09ec already in progress: creating tablet
W20251028 03:00:04.150066 8083 leader_election.cc:343] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: d60fc654f6af44b290d46235082a4113
W20251028 03:00:04.150215 8083 leader_election.cc:343] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:04.150296 8083 leader_election.cc:343] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 8720ea8c68904d57a3fc5c0e94d338ca
W20251028 03:00:04.150367 8083 leader_election.cc:343] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.150403 8083 leader_election.cc:304] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:04.150630 8083 leader_election.cc:304] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.152674 8552 tablet_replica.cc:333] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.152840 8552 raft_consensus.cc:2243] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.152969 8552 raft_consensus.cc:2272] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.153065 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8720ea8c68904d57a3fc5c0e94d338ca" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.153410 8552 ts_tablet_manager.cc:1916] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.153414 10335 raft_consensus.cc:2749] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.153532 10335 raft_consensus.cc:2804] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.153570 10335 raft_consensus.cc:493] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.153605 10335 raft_consensus.cc:3060] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.154696 10335 raft_consensus.cc:515] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.155051 10335 leader_election.cc:290] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.156010 8552 ts_tablet_manager.cc:1929] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.156100 8552 log.cc:1199] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/8665d2e933fa423d81a1b32f2ce8629b
I20251028 03:00:04.156360 8552 ts_tablet_manager.cc:1950] T 8665d2e933fa423d81a1b32f2ce8629b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.157027 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 8665d2e933fa423d81a1b32f2ce8629b (table default.Table_139255bc4fb74fae8fb86563b54826bd [id=dc3721126b09466ab5efdb8bd4b46d89]) successfully deleted
W20251028 03:00:04.157305 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet da46096a2001470ba1bd1c9660e4b0c5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da46096a2001470ba1bd1c9660e4b0c5 already in progress: creating tablet
W20251028 03:00:04.157622 8083 leader_election.cc:343] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.157703 8083 leader_election.cc:304] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:04.158310 8545 tablet_bootstrap.cc:492] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.158394 8545 ts_tablet_manager.cc:1403] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20251028 03:00:04.158447 8546 tablet_bootstrap.cc:492] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.158525 8546 ts_tablet_manager.cc:1403] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251028 03:00:04.158895 8545 raft_consensus.cc:359] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.158990 8545 raft_consensus.cc:385] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.159017 8545 raft_consensus.cc:740] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.158973 8546 raft_consensus.cc:359] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.159067 8546 raft_consensus.cc:385] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.159094 8546 raft_consensus.cc:740] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.159129 8545 consensus_queue.cc:260] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.159204 8546 consensus_queue.cc:260] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.159417 8545 ts_tablet_manager.cc:1434] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.159487 8546 ts_tablet_manager.cc:1434] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:04.159607 8545 tablet_bootstrap.cc:492] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.160669 8545 tablet_bootstrap.cc:654] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.160650 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6dd563e13e624d4bb88950b46d480876" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:04.160890 10253 raft_consensus.cc:2749] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.161123 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6dd563e13e624d4bb88950b46d480876" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.161226 8420 raft_consensus.cc:3060] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.162266 8420 raft_consensus.cc:2468] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
W20251028 03:00:04.162956 8083 leader_election.cc:343] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.163101 8083 leader_election.cc:304] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.163424 10335 raft_consensus.cc:2804] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.163443 8546 tablet_bootstrap.cc:492] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.163511 10335 raft_consensus.cc:697] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:04.163645 10335 consensus_queue.cc:237] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.166188 8546 tablet_bootstrap.cc:654] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.168223 8546 tablet_bootstrap.cc:492] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.168431 8546 ts_tablet_manager.cc:1403] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20251028 03:00:04.169000 8546 raft_consensus.cc:359] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.169178 8546 raft_consensus.cc:385] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.169256 8546 raft_consensus.cc:740] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.169423 8546 consensus_queue.cc:260] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.170979 10253 raft_consensus.cc:493] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.171070 10253 raft_consensus.cc:515] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.171473 10253 leader_election.cc:290] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.171914 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8e202e53053248a7a9cee6a217999489" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:04.172273 8083 leader_election.cc:343] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.173125 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_67078ec90ffe4bb7bbf75491a56d0b47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:04.173348 8546 ts_tablet_manager.cc:1434] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
W20251028 03:00:04.173511 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67078ec90ffe4bb7bbf75491a56d0b47 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:04.174330 8083 leader_election.cc:343] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 8e202e53053248a7a9cee6a217999489
I20251028 03:00:04.174399 8083 leader_election.cc:304] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:04.174510 8544 tablet_bootstrap.cc:492] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.174600 8544 ts_tablet_manager.cc:1403] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.000s
I20251028 03:00:04.174788 8546 tablet_bootstrap.cc:492] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.175074 8544 raft_consensus.cc:359] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.175185 8544 raft_consensus.cc:385] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.175225 8544 raft_consensus.cc:740] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.175398 8544 consensus_queue.cc:260] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.175710 8544 ts_tablet_manager.cc:1434] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:04.175899 8544 tablet_bootstrap.cc:492] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.176302 8546 tablet_bootstrap.cc:654] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.177518 8546 tablet_bootstrap.cc:492] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.177600 8546 ts_tablet_manager.cc:1403] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 03:00:04.178120 8546 raft_consensus.cc:359] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.178220 8546 raft_consensus.cc:385] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.178257 8546 raft_consensus.cc:740] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.178376 8546 consensus_queue.cc:260] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.178678 8546 ts_tablet_manager.cc:1434] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.178881 8546 tablet_bootstrap.cc:492] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.179347 8545 tablet_bootstrap.cc:492] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.179464 8545 ts_tablet_manager.cc:1403] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20251028 03:00:04.180032 8545 raft_consensus.cc:359] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.180137 8545 raft_consensus.cc:385] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.180169 8545 raft_consensus.cc:740] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.180284 8545 consensus_queue.cc:260] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.180474 8546 tablet_bootstrap.cc:654] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.180596 8545 ts_tablet_manager.cc:1434] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:04.181561 8544 tablet_bootstrap.cc:654] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.183338 7967 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1692c48a1cb243039125140ab7932f6b" } new_table_name: "default.Table_4b231be36bff4ad1b7a63b650b3a3ae1" modify_external_catalogs: true
I20251028 03:00:04.183710 8545 tablet_bootstrap.cc:492] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 03:00:04.184439 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 782aed99e4b2455893de8896aa456ce9 consensus state has no leader
I20251028 03:00:04.184679 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_829c6eb42be14ebbb7b97fec16a89ba6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:04.184924 8545 tablet_bootstrap.cc:654] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:04.185065 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_829c6eb42be14ebbb7b97fec16a89ba6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.186296 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9a81841c22ae49c0a17931d90407accc
I20251028 03:00:04.186563 7966 catalog_manager.cc:6719] ReplaceTablet: tablet 9a81841c22ae49c0a17931d90407accc deleted and replaced by tablet 64ff84e80f4647d9a7c6eea42d11e56e
I20251028 03:00:04.186686 7966 master_service.cc:954] ReplaceTablet: received request to replace tablet ee2fd98744ff44028f36bb55e6eba51e from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:04.186803 8387 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a81841c22ae49c0a17931d90407accc with delete_type TABLET_DATA_DELETED (replaced by tablet 64ff84e80f4647d9a7c6eea42d11e56e) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.186848 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a81841c22ae49c0a17931d90407accc with delete_type TABLET_DATA_DELETED (replaced by tablet 64ff84e80f4647d9a7c6eea42d11e56e) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.186985 8551 tablet_replica.cc:333] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.187034 8552 tablet_replica.cc:333] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.187182 8552 raft_consensus.cc:2243] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.187330 8552 raft_consensus.cc:2272] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.187400 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 9a81841c22ae49c0a17931d90407accc with delete_type TABLET_DATA_DELETED (replaced by tablet 64ff84e80f4647d9a7c6eea42d11e56e) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.187774 8552 ts_tablet_manager.cc:1916] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.188210 7965 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_18a5674a59e34ce4b3bf54efce8a8a44" } modify_external_catalogs: true
I20251028 03:00:04.188323 7965 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_18a5674a59e34ce4b3bf54efce8a8a44" } modify_external_catalogs: true
I20251028 03:00:04.188306 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e3764076fbd04d449a8b8249a205f2cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.188719 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e3764076fbd04d449a8b8249a205f2cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.189172 8550 tablet_replica.cc:333] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.189348 8550 raft_consensus.cc:2243] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.189488 8550 raft_consensus.cc:2272] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.189967 8550 ts_tablet_manager.cc:1916] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.190136 8552 ts_tablet_manager.cc:1929] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.190202 8552 log.cc:1199] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/9a81841c22ae49c0a17931d90407accc
I20251028 03:00:04.190145 7969 catalog_manager.cc:5649] T f6818c3f2dfa48448bb2d9b38ed0dc19 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:04.190372 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet f6818c3f2dfa48448bb2d9b38ed0dc19 (table default.Table_73967d33588f4f02ab5e0010826ec5f6 [id=24424b561e55419cbc0d32089a83e21a]). Expected version 1 got 0
I20251028 03:00:04.190457 8552 ts_tablet_manager.cc:1950] T 9a81841c22ae49c0a17931d90407accc P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.190557 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet e3b33c4d189247a89878b6a8e1fcbac0 (table default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]). Expected version 1 got 0
I20251028 03:00:04.190819 7969 catalog_manager.cc:5649] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:04.191025 7969 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 58268ac590f3437a942dcafd927a5496 (table default.Table_0b7b98e5884a48de833ab1fbbad97f08 [id=fdb289a561dd4c6497173ef43e2e9ccd]). Expected version 1 got 0
I20251028 03:00:04.191526 9958 raft_consensus.cc:493] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.191619 9958 raft_consensus.cc:515] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.191687 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 9a81841c22ae49c0a17931d90407accc (table default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]) successfully deleted
I20251028 03:00:04.191994 9958 leader_election.cc:290] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.192076 8386 tablet_service.cc:1505] Processing CreateTablet for tablet df88e048b2ee470a8af37a2aae6a8fc9 (DEFAULT_TABLE table=default.Table_ff74a0ebc6284f5f813ca17bd15504d4 [id=3c7537ee6fd34f9cbc7bf304ca7b5a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.192365 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df88e048b2ee470a8af37a2aae6a8fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.192515 8265 tablet_service.cc:1505] Processing CreateTablet for tablet df88e048b2ee470a8af37a2aae6a8fc9 (DEFAULT_TABLE table=default.Table_ff74a0ebc6284f5f813ca17bd15504d4 [id=3c7537ee6fd34f9cbc7bf304ca7b5a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.192575 8252 tablet_service.cc:1505] Processing CreateTablet for tablet aa0e9627a59c4eadb7901bc159281880 (DEFAULT_TABLE table=default.Table_dfa4ca07c0c54cef8189b4c26ac43c9a [id=517c0fd4f88c46b9b6ea59f5f0d73168]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.192775 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df88e048b2ee470a8af37a2aae6a8fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.192938 8261 tablet_service.cc:1505] Processing CreateTablet for tablet d2d3563058164f80aaa72ddf584671e5 (DEFAULT_TABLE table=default.Table_0e2fb96fdd1e49429c3f2e5dc91bf766 [id=808b3cabb15d407db18594aa25bc3978]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.193147 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2d3563058164f80aaa72ddf584671e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.193951 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 3704e560d7314284a6af2852a756348e (DEFAULT_TABLE table=default.Table_4dc821cafb8143f2aff8dab1d512a544 [id=abe9ea5a664b46dbac01a759c8d2eab8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.194486 8397 tablet_service.cc:1505] Processing CreateTablet for tablet aa0e9627a59c4eadb7901bc159281880 (DEFAULT_TABLE table=default.Table_dfa4ca07c0c54cef8189b4c26ac43c9a [id=517c0fd4f88c46b9b6ea59f5f0d73168]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.194784 8396 tablet_service.cc:1505] Processing CreateTablet for tablet d2d3563058164f80aaa72ddf584671e5 (DEFAULT_TABLE table=default.Table_0e2fb96fdd1e49429c3f2e5dc91bf766 [id=808b3cabb15d407db18594aa25bc3978]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.195021 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 3704e560d7314284a6af2852a756348e (DEFAULT_TABLE table=default.Table_4dc821cafb8143f2aff8dab1d512a544 [id=abe9ea5a664b46dbac01a759c8d2eab8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.195194 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa0e9627a59c4eadb7901bc159281880. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.195363 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3704e560d7314284a6af2852a756348e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.195485 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 3704e560d7314284a6af2852a756348e (DEFAULT_TABLE table=default.Table_4dc821cafb8143f2aff8dab1d512a544 [id=abe9ea5a664b46dbac01a759c8d2eab8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.195739 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3704e560d7314284a6af2852a756348e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.195828 8125 tablet_service.cc:1505] Processing CreateTablet for tablet b53ba38b16c84b199490ce8b93316b85 (DEFAULT_TABLE table=default.Table_80603ad4c2bc4966bbc952208ce043e2 [id=a222e22ca83a46f793259accf7b4bfbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.196084 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b53ba38b16c84b199490ce8b93316b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.197528 8122 tablet_service.cc:1505] Processing CreateTablet for tablet df88e048b2ee470a8af37a2aae6a8fc9 (DEFAULT_TABLE table=default.Table_ff74a0ebc6284f5f813ca17bd15504d4 [id=3c7537ee6fd34f9cbc7bf304ca7b5a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.197744 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df88e048b2ee470a8af37a2aae6a8fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.198019 8121 tablet_service.cc:1505] Processing CreateTablet for tablet aa0e9627a59c4eadb7901bc159281880 (DEFAULT_TABLE table=default.Table_dfa4ca07c0c54cef8189b4c26ac43c9a [id=517c0fd4f88c46b9b6ea59f5f0d73168]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.198232 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa0e9627a59c4eadb7901bc159281880. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.200146 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa0e9627a59c4eadb7901bc159281880. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.200312 8118 tablet_service.cc:1505] Processing CreateTablet for tablet d2d3563058164f80aaa72ddf584671e5 (DEFAULT_TABLE table=default.Table_0e2fb96fdd1e49429c3f2e5dc91bf766 [id=808b3cabb15d407db18594aa25bc3978]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.200484 9595 raft_consensus.cc:493] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.200531 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2d3563058164f80aaa72ddf584671e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.200565 9595 raft_consensus.cc:515] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.200935 9595 leader_election.cc:290] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.201987 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2d3563058164f80aaa72ddf584671e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.202314 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3704e560d7314284a6af2852a756348e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.202615 8120 tablet_service.cc:1505] Processing CreateTablet for tablet acd1d09b52ae4b80b0cafeb35d196c2c (DEFAULT_TABLE table=default.Table_5a9578228e3b45b9a6d9a39b4d440b81 [id=08529f97efe848d68683965630300299]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.202842 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acd1d09b52ae4b80b0cafeb35d196c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.203611 10253 raft_consensus.cc:2749] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.203786 10253 raft_consensus.cc:493] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.203859 10253 raft_consensus.cc:515] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.204205 10253 leader_election.cc:290] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.204278 10253 raft_consensus.cc:493] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.204329 10253 raft_consensus.cc:515] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.204654 10253 leader_election.cc:290] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.204878 8127 tablet_service.cc:1505] Processing CreateTablet for tablet dfa009c0c5a04b6ea0501fd81ebb1890 (DEFAULT_TABLE table=default.Table_c01a7348790a498eaa6a628519733f77 [id=3a7c7ece14984ad283d189d086034415]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.205106 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfa009c0c5a04b6ea0501fd81ebb1890. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.205430 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a60624333d43f1b701036b7e9dfdb0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.205462 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58268ac590f3437a942dcafd927a5496" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.205523 8421 raft_consensus.cc:2468] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:04.205787 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a60624333d43f1b701036b7e9dfdb0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:04.205827 8083 leader_election.cc:343] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:04.206326 8083 leader_election.cc:343] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.206642 8083 leader_election.cc:304] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.206868 10253 raft_consensus.cc:2804] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.206920 10253 raft_consensus.cc:493] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.206970 10253 raft_consensus.cc:3060] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.207134 8550 ts_tablet_manager.cc:1929] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.207194 8550 log.cc:1199] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/9a81841c22ae49c0a17931d90407accc
I20251028 03:00:04.207376 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58268ac590f3437a942dcafd927a5496" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.207489 8418 raft_consensus.cc:2468] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:04.207542 8550 ts_tablet_manager.cc:1950] T 9a81841c22ae49c0a17931d90407accc P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.207878 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e4d980617994db4a188a98bb31eff92" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.207934 10253 raft_consensus.cc:515] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.207986 8420 raft_consensus.cc:2468] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
W20251028 03:00:04.208019 8220 leader_election.cc:343] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: bedb4fd3b46f4453b194cfaea0c3c701
I20251028 03:00:04.208245 10253 leader_election.cc:290] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.208261 8220 leader_election.cc:304] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:04.208436 9958 raft_consensus.cc:2804] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.208491 9958 raft_consensus.cc:493] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.208526 9958 raft_consensus.cc:3060] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.208731 8083 leader_election.cc:304] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.209188 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 9a81841c22ae49c0a17931d90407accc (table default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]) successfully deleted
I20251028 03:00:04.209506 9958 raft_consensus.cc:515] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.209676 10253 raft_consensus.cc:2804] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.209757 10253 raft_consensus.cc:493] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.209810 10253 raft_consensus.cc:3060] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.209911 9958 leader_election.cc:290] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.210620 10253 raft_consensus.cc:515] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.211097 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a60624333d43f1b701036b7e9dfdb0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.211153 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a60624333d43f1b701036b7e9dfdb0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:04.211212 8420 raft_consensus.cc:3060] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.211470 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58268ac590f3437a942dcafd927a5496" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:04.211486 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e4d980617994db4a188a98bb31eff92" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.211570 8419 raft_consensus.cc:3060] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:04.211804 8083 leader_election.cc:343] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:04.211928 8083 leader_election.cc:343] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.212044 10253 leader_election.cc:290] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.212260 8420 raft_consensus.cc:2468] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:04.212442 8419 raft_consensus.cc:2468] T 4e4d980617994db4a188a98bb31eff92 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:04.212524 8083 leader_election.cc:304] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.212632 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "58268ac590f3437a942dcafd927a5496" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.212713 10253 raft_consensus.cc:2804] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.212723 8418 raft_consensus.cc:3060] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.212785 10253 raft_consensus.cc:697] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:04.212898 10253 consensus_queue.cc:237] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.213241 8220 leader_election.cc:304] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:04.213409 9958 raft_consensus.cc:2804] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.213483 9958 raft_consensus.cc:697] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:04.213639 8418 raft_consensus.cc:2468] T 58268ac590f3437a942dcafd927a5496 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:04.213631 9958 consensus_queue.cc:237] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.213929 8083 leader_election.cc:304] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.214448 8551 raft_consensus.cc:2243] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.214676 8551 raft_consensus.cc:2272] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.215095 8551 ts_tablet_manager.cc:1916] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.215765 9595 consensus_queue.cc:1048] T a38ef10c145a4d00a26265a9dee5c744 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.217234 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e4d980617994db4a188a98bb31eff92" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:04.217329 8143 raft_consensus.cc:3060] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.217639 8148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bedb4fd3b46f4453b194cfaea0c3c701" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.217774 8148 raft_consensus.cc:2393] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 03:00:04.217820 8551 ts_tablet_manager.cc:1929] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.217885 8551 log.cc:1199] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/9a81841c22ae49c0a17931d90407accc
I20251028 03:00:04.218134 8551 ts_tablet_manager.cc:1950] T 9a81841c22ae49c0a17931d90407accc P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.218353 8143 raft_consensus.cc:2468] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:04.218426 8147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4e4d980617994db4a188a98bb31eff92" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.218518 8147 raft_consensus.cc:2376] T 4e4d980617994db4a188a98bb31eff92 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1. Re-sending same reply.
I20251028 03:00:04.218765 7967 master_service.cc:954] ReplaceTablet: received request to replace tablet 67f3b694dcc5465ca2c8c5535e7efea6 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:04.219151 8252 tablet_service.cc:1505] Processing CreateTablet for tablet b53ba38b16c84b199490ce8b93316b85 (DEFAULT_TABLE table=default.Table_80603ad4c2bc4966bbc952208ce043e2 [id=a222e22ca83a46f793259accf7b4bfbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.219461 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b53ba38b16c84b199490ce8b93316b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.220471 8256 tablet_service.cc:1505] Processing CreateTablet for tablet acd1d09b52ae4b80b0cafeb35d196c2c (DEFAULT_TABLE table=default.Table_5a9578228e3b45b9a6d9a39b4d440b81 [id=08529f97efe848d68683965630300299]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.220706 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acd1d09b52ae4b80b0cafeb35d196c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.220968 8218 leader_election.cc:304] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.221035 8396 tablet_service.cc:1505] Processing CreateTablet for tablet b53ba38b16c84b199490ce8b93316b85 (DEFAULT_TABLE table=default.Table_80603ad4c2bc4966bbc952208ce043e2 [id=a222e22ca83a46f793259accf7b4bfbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.221271 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b53ba38b16c84b199490ce8b93316b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.221752 8261 tablet_service.cc:1505] Processing CreateTablet for tablet dfa009c0c5a04b6ea0501fd81ebb1890 (DEFAULT_TABLE table=default.Table_c01a7348790a498eaa6a628519733f77 [id=3a7c7ece14984ad283d189d086034415]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.221952 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfa009c0c5a04b6ea0501fd81ebb1890. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.223372 8553 raft_consensus.cc:3060] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.224267 8553 raft_consensus.cc:2749] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.224740 8397 tablet_service.cc:1505] Processing CreateTablet for tablet acd1d09b52ae4b80b0cafeb35d196c2c (DEFAULT_TABLE table=default.Table_5a9578228e3b45b9a6d9a39b4d440b81 [id=08529f97efe848d68683965630300299]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.224781 8387 tablet_service.cc:1505] Processing CreateTablet for tablet dfa009c0c5a04b6ea0501fd81ebb1890 (DEFAULT_TABLE table=default.Table_c01a7348790a498eaa6a628519733f77 [id=3a7c7ece14984ad283d189d086034415]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.225008 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acd1d09b52ae4b80b0cafeb35d196c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.225186 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfa009c0c5a04b6ea0501fd81ebb1890. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.225849 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 9a81841c22ae49c0a17931d90407accc (table default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]) successfully deleted
I20251028 03:00:04.226686 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 32efb7b57678451f97abea5ea61da87a
I20251028 03:00:04.226831 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 66d385494c7a4fb8a6501519925be7d4
I20251028 03:00:04.227170 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5f779c29ab284a649387eeddd3e62662"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.227630 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f779c29ab284a649387eeddd3e62662 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.228708 7962 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet e3b33c4d189247a89878b6a8e1fcbac0 (table default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c]). Expected version 1 got 0
I20251028 03:00:04.229172 7968 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_77a37bbdfa4f41458e6b50fb213850bc [id=5f61b3c12ed647968fc2932aa24a13f3] alter complete (version 2)
I20251028 03:00:04.229523 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_be36f6bb04274166a890026a70c7dbf7" } new_table_name: "default.Table_d33abe2095b64dd499649ef1e6a64943" modify_external_catalogs: true
I20251028 03:00:04.229573 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_efbcb24780904b469cfe628420a6addd" } new_table_name: "default.Table_41dffdb5f3d848db840c0e7174dc3eac" modify_external_catalogs: true
I20251028 03:00:04.230340 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 32efb7b57678451f97abea5ea61da87a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.230583 7961 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 16123c06b3d94acfa11145963bd31b6b
W20251028 03:00:04.230856 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 32efb7b57678451f97abea5ea61da87a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32efb7b57678451f97abea5ea61da87a already in progress: creating tablet
I20251028 03:00:04.231443 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet ee1d686a0e4c47f6b214224047a55747 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.231595 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet 16123c06b3d94acfa11145963bd31b6b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.231729 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 16123c06b3d94acfa11145963bd31b6b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.231791 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 32efb7b57678451f97abea5ea61da87a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.231932 8551 tablet_replica.cc:333] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.232030 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b787f8f024745b297b2ebaa456cbcbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.232085 8551 raft_consensus.cc:2243] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.232152 8552 tablet_replica.cc:333] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.232275 8552 raft_consensus.cc:2243] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.232372 8552 raft_consensus.cc:2272] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.232197 8551 raft_consensus.cc:2272] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.233572 8552 ts_tablet_manager.cc:1916] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.234596 8551 ts_tablet_manager.cc:1916] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.236567 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet bedb4fd3b46f4453b194cfaea0c3c701 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.236723 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet ee1d686a0e4c47f6b214224047a55747 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.236920 8552 ts_tablet_manager.cc:1929] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.237007 8552 log.cc:1199] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/4b787f8f024745b297b2ebaa456cbcbf
I20251028 03:00:04.237237 8552 ts_tablet_manager.cc:1950] T 4b787f8f024745b297b2ebaa456cbcbf P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.237866 8552 tablet_replica.cc:333] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.238016 8552 raft_consensus.cc:2243] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.238131 8552 raft_consensus.cc:2272] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.238489 8551 ts_tablet_manager.cc:1929] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.238588 8551 log.cc:1199] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ee1d686a0e4c47f6b214224047a55747
I20251028 03:00:04.238831 8551 ts_tablet_manager.cc:1950] T ee1d686a0e4c47f6b214224047a55747 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.238973 10253 raft_consensus.cc:2804] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.239082 10253 raft_consensus.cc:697] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:04.239732 10253 consensus_queue.cc:237] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.240216 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 66d385494c7a4fb8a6501519925be7d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.241254 10335 raft_consensus.cc:493] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.241339 10335 raft_consensus.cc:515] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.241660 10335 leader_election.cc:290] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.242405 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 66d385494c7a4fb8a6501519925be7d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.242625 7963 catalog_manager.cc:5649] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 03:00:04.242852 8552 ts_tablet_manager.cc:1916] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.242911 7963 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 4f90e3c3524e4498aa6c4bfc5304d44b (table default.Table_42f89d7fccbf4c738293e9796be235a0 [id=6f3e689a22ae4160ae7537d955c06a90]). Expected version 1 got 0
I20251028 03:00:04.243059 7963 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet a38ef10c145a4d00a26265a9dee5c744 (table default.Table_8d72f9b8a87847d0a40f66c2bcdba651 [id=ba8b6a629ebc489abe7043890455765a]). Expected version 1 got 0
I20251028 03:00:04.227180 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_3ba22f248ad04933b79a9c84122d09a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.243889 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ba22f248ad04933b79a9c84122d09a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20251028 03:00:04.240072 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 16123c06b3d94acfa11145963bd31b6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16123c06b3d94acfa11145963bd31b6b already in progress: creating tablet
I20251028 03:00:04.245525 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 4b787f8f024745b297b2ebaa456cbcbf (table default.Table_35aa9cb3afdc4a149341fa51b840258f [id=a05d3a9fbe8c4278bcab247dcd663653]) successfully deleted
I20251028 03:00:04.245829 8552 ts_tablet_manager.cc:1929] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.245894 8552 log.cc:1199] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/bedb4fd3b46f4453b194cfaea0c3c701
I20251028 03:00:04.246109 8552 ts_tablet_manager.cc:1950] T bedb4fd3b46f4453b194cfaea0c3c701 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.246722 8552 tablet_replica.cc:333] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.246876 8552 raft_consensus.cc:2243] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.246977 8552 raft_consensus.cc:2272] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.247256 8397 tablet_service.cc:1505] Processing CreateTablet for tablet 4f91555da1bd4b9c971db13f461c1427 (DEFAULT_TABLE table=default.Table_93cfe7776e94425c91cf3b046fcd1923 [id=87631fab5ece4358ae75d866030d31a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.247390 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 4f91555da1bd4b9c971db13f461c1427 (DEFAULT_TABLE table=default.Table_93cfe7776e94425c91cf3b046fcd1923 [id=87631fab5ece4358ae75d866030d31a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.247565 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f91555da1bd4b9c971db13f461c1427. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.247643 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f91555da1bd4b9c971db13f461c1427. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.250095 8551 tablet_replica.cc:333] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.250348 8551 raft_consensus.cc:2243] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.250494 8551 raft_consensus.cc:2272] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.251506 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fec0adb79d04469b90c1ea2a5e9ba5b" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.251773 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 8720ea8c68904d57a3fc5c0e94d338ca with delete_type TABLET_DATA_DELETED (replaced by tablet 30232298155542ecb41f5a1b19b16ffb) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.251969 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 3435803c6167447a8b3afba6c0f3511b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.252056 8115 tablet_service.cc:1552] Processing DeleteTablet for tablet 8e202e53053248a7a9cee6a217999489 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.252091 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b0c593a3b96434db9ac42d260fc6997 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.252146 8122 tablet_service.cc:1552] Processing DeleteTablet for tablet d60fc654f6af44b290d46235082a4113 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d42a5987fa540ce86afdbbe1922896e) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.252169 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b787f8f024745b297b2ebaa456cbcbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.252313 8115 tablet_service.cc:1552] Processing DeleteTablet for tablet 16123c06b3d94acfa11145963bd31b6b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.252329 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet d5c3b02db24d4dc48260b42841844938 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:04.252429 8083 leader_election.cc:343] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 2fec0adb79d04469b90c1ea2a5e9ba5b
W20251028 03:00:04.252557 8083 leader_election.cc:343] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.252602 8083 leader_election.cc:304] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:04.252576 8124 tablet_service.cc:1505] Processing CreateTablet for tablet 4f91555da1bd4b9c971db13f461c1427 (DEFAULT_TABLE table=default.Table_93cfe7776e94425c91cf3b046fcd1923 [id=87631fab5ece4358ae75d866030d31a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.252830 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f91555da1bd4b9c971db13f461c1427. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.252899 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet 32efb7b57678451f97abea5ea61da87a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:03 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.253005 8550 tablet_replica.cc:333] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.253021 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet 66d385494c7a4fb8a6501519925be7d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.253103 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet e369dbabc2c04ad5ab17bbd4225127c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.253140 8550 raft_consensus.cc:2243] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.253185 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet bedb4fd3b46f4453b194cfaea0c3c701 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.253242 8550 raft_consensus.cc:2272] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.253396 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet a4bcdc8a0c554b7cae81d7993e4f2861 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.253818 8550 ts_tablet_manager.cc:1916] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.255146 10335 raft_consensus.cc:2749] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.255528 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ee1d686a0e4c47f6b214224047a55747 (table default.Table_6ebfbcbecf7c4975ac638a06859f32d2 [id=27a9fc6733374573b6370d76769a7653]) successfully deleted
I20251028 03:00:04.256806 8550 ts_tablet_manager.cc:1929] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.256871 8550 log.cc:1199] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8720ea8c68904d57a3fc5c0e94d338ca
I20251028 03:00:04.257092 8550 ts_tablet_manager.cc:1950] T 8720ea8c68904d57a3fc5c0e94d338ca P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.257373 8550 tablet_replica.cc:333] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.257504 8550 raft_consensus.cc:2243] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
W20251028 03:00:04.255693 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 16123c06b3d94acfa11145963bd31b6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16123c06b3d94acfa11145963bd31b6b already in progress: creating tablet
I20251028 03:00:04.257607 8550 raft_consensus.cc:2272] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.256652 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 67f3b694dcc5465ca2c8c5535e7efea6
W20251028 03:00:04.257781 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 32efb7b57678451f97abea5ea61da87a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32efb7b57678451f97abea5ea61da87a already in progress: creating tablet
I20251028 03:00:04.257913 7967 catalog_manager.cc:6719] ReplaceTablet: tablet 67f3b694dcc5465ca2c8c5535e7efea6 deleted and replaced by tablet b3b934bbc7fe4581a75b4b3b01e8d66f
I20251028 03:00:04.257972 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet bedb4fd3b46f4453b194cfaea0c3c701 (table default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]) successfully deleted
I20251028 03:00:04.257987 8550 ts_tablet_manager.cc:1916] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.259132 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 67f3b694dcc5465ca2c8c5535e7efea6 with delete_type TABLET_DATA_DELETED (replaced by tablet b3b934bbc7fe4581a75b4b3b01e8d66f) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.259234 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8720ea8c68904d57a3fc5c0e94d338ca (table default.Table_79ced5936022425c91473064b87986d3 [id=375c6744f94644199d8d6f08f21ea021]) successfully deleted
I20251028 03:00:04.259339 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 67f3b694dcc5465ca2c8c5535e7efea6 with delete_type TABLET_DATA_DELETED (replaced by tablet b3b934bbc7fe4581a75b4b3b01e8d66f) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.259737 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 67f3b694dcc5465ca2c8c5535e7efea6 with delete_type TABLET_DATA_DELETED (replaced by tablet b3b934bbc7fe4581a75b4b3b01e8d66f) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.260121 8551 ts_tablet_manager.cc:1916] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.260129 8552 ts_tablet_manager.cc:1916] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.262530 8552 ts_tablet_manager.cc:1929] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.262603 8552 log.cc:1199] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/ee1d686a0e4c47f6b214224047a55747
I20251028 03:00:04.262813 8552 ts_tablet_manager.cc:1950] T ee1d686a0e4c47f6b214224047a55747 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.263501 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet ee1d686a0e4c47f6b214224047a55747 (table default.Table_6ebfbcbecf7c4975ac638a06859f32d2 [id=27a9fc6733374573b6370d76769a7653]) successfully deleted
I20251028 03:00:04.264364 10422 tablet.cc:1722] T b93581f0eb7e499597c4e5d2f6c058aa P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:04.264981 8552 tablet_replica.cc:333] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.265121 10423 tablet.cc:1722] T b93581f0eb7e499597c4e5d2f6c058aa P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:04.265216 8552 raft_consensus.cc:2243] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.265399 8552 raft_consensus.cc:2272] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.265795 8551 ts_tablet_manager.cc:1929] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.265836 8552 ts_tablet_manager.cc:1916] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.265990 8550 ts_tablet_manager.cc:1929] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.266072 8550 log.cc:1199] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3435803c6167447a8b3afba6c0f3511b
I20251028 03:00:04.266306 8550 ts_tablet_manager.cc:1950] T 3435803c6167447a8b3afba6c0f3511b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.266629 8550 tablet_replica.cc:333] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.266760 8550 raft_consensus.cc:2243] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.266759 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3435803c6167447a8b3afba6c0f3511b (table default.Table_e40f83dab24e4e83b634a88cfe208817 [id=b987718414e246efb3353a3c8bd35f2f]) successfully deleted
I20251028 03:00:04.266871 8550 raft_consensus.cc:2272] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.267010 8551 log.cc:1199] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/66d385494c7a4fb8a6501519925be7d4
I20251028 03:00:04.267500 8551 ts_tablet_manager.cc:1950] T 66d385494c7a4fb8a6501519925be7d4 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.268819 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 66d385494c7a4fb8a6501519925be7d4 (table default.Table_18a5674a59e34ce4b3bf54efce8a8a44 [id=20ee524d16f941159b5c44e85ed885f6]) successfully deleted
I20251028 03:00:04.268998 8552 ts_tablet_manager.cc:1929] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.269057 8552 log.cc:1199] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/66d385494c7a4fb8a6501519925be7d4
I20251028 03:00:04.269567 8552 ts_tablet_manager.cc:1950] T 66d385494c7a4fb8a6501519925be7d4 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.270459 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 66d385494c7a4fb8a6501519925be7d4 (table default.Table_18a5674a59e34ce4b3bf54efce8a8a44 [id=20ee524d16f941159b5c44e85ed885f6]) successfully deleted
I20251028 03:00:04.270810 10187 tablet.cc:1722] T b93581f0eb7e499597c4e5d2f6c058aa P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20251028 03:00:04.271027 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 67f3b694dcc5465ca2c8c5535e7efea6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67f3b694dcc5465ca2c8c5535e7efea6 already in progress: creating tablet
I20251028 03:00:04.271795 8550 ts_tablet_manager.cc:1916] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.279208 8546 tablet_bootstrap.cc:492] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.279439 8546 ts_tablet_manager.cc:1403] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.101s user 0.000s sys 0.002s
I20251028 03:00:04.279960 8546 raft_consensus.cc:359] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.280063 8546 raft_consensus.cc:385] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.280094 8546 raft_consensus.cc:740] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.280198 8546 consensus_queue.cc:260] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.280478 8546 ts_tablet_manager.cc:1434] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.280675 8546 tablet_bootstrap.cc:492] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.281877 8546 tablet_bootstrap.cc:654] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.284607 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet a52b2fe9282b486682a961fc2136557c from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:04.284832 8550 ts_tablet_manager.cc:1929] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.284919 8550 log.cc:1199] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/8e202e53053248a7a9cee6a217999489
I20251028 03:00:04.285140 8550 ts_tablet_manager.cc:1950] T 8e202e53053248a7a9cee6a217999489 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 03:00:04.285161 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 67f3b694dcc5465ca2c8c5535e7efea6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67f3b694dcc5465ca2c8c5535e7efea6 already in progress: creating tablet
I20251028 03:00:04.285476 8550 tablet_replica.cc:333] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.285596 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 8e202e53053248a7a9cee6a217999489 (table default.Table_01d864857be34675882e1be7ba5a9a7c [id=32dae7dc982547979850fa8ad8a80de9]) successfully deleted
I20251028 03:00:04.285624 8550 raft_consensus.cc:2243] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.285743 8550 raft_consensus.cc:2272] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.286036 8546 tablet_bootstrap.cc:492] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.286091 8550 ts_tablet_manager.cc:1916] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.286135 8546 ts_tablet_manager.cc:1403] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20251028 03:00:04.286609 8546 raft_consensus.cc:359] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.286733 8546 raft_consensus.cc:385] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.286784 8546 raft_consensus.cc:740] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.286924 8546 consensus_queue.cc:260] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.287215 8546 ts_tablet_manager.cc:1434] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.287457 8546 tablet_bootstrap.cc:492] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.289227 8546 tablet_bootstrap.cc:654] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.290341 8546 tablet_bootstrap.cc:492] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.290424 8546 ts_tablet_manager.cc:1403] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251028 03:00:04.290946 8546 raft_consensus.cc:359] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.291044 8546 raft_consensus.cc:385] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.291077 8546 raft_consensus.cc:740] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.291193 8546 consensus_queue.cc:260] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.291482 8546 ts_tablet_manager.cc:1434] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20251028 03:00:04.291531 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bdd82979f1b9436fa59c2b92e0b06ec3 consensus state has no leader
I20251028 03:00:04.291656 8546 tablet_bootstrap.cc:492] T 58268ac590f3437a942dcafd927a5496 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.288048 8550 ts_tablet_manager.cc:1929] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.292817 8550 log.cc:1199] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/3b0c593a3b96434db9ac42d260fc6997
I20251028 03:00:04.293061 8550 ts_tablet_manager.cc:1950] T 3b0c593a3b96434db9ac42d260fc6997 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.293212 8544 tablet_bootstrap.cc:492] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.293309 8544 ts_tablet_manager.cc:1403] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.117s user 0.001s sys 0.000s
I20251028 03:00:04.293639 8550 tablet_replica.cc:333] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.293783 8550 raft_consensus.cc:2243] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.293743 8544 raft_consensus.cc:359] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.293851 8544 raft_consensus.cc:385] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.293869 8550 raft_consensus.cc:2272] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.293891 8544 raft_consensus.cc:740] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.294046 8544 consensus_queue.cc:260] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.294404 8544 ts_tablet_manager.cc:1434] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.294615 8544 tablet_bootstrap.cc:492] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.294651 8545 tablet_bootstrap.cc:492] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.294765 8545 ts_tablet_manager.cc:1403] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.111s user 0.002s sys 0.000s
I20251028 03:00:04.295378 8545 raft_consensus.cc:359] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.295512 8545 raft_consensus.cc:385] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.295552 8545 raft_consensus.cc:740] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.295696 8545 consensus_queue.cc:260] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.296048 8545 ts_tablet_manager.cc:1434] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.296278 8545 tablet_bootstrap.cc:492] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.296890 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 3b0c593a3b96434db9ac42d260fc6997 (table default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]) successfully deleted
I20251028 03:00:04.297394 8545 tablet_bootstrap.cc:654] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.298627 8544 tablet_bootstrap.cc:654] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.298862 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet f6818c3f2dfa48448bb2d9b38ed0dc19 (table default.Table_73967d33588f4f02ab5e0010826ec5f6 [id=24424b561e55419cbc0d32089a83e21a]). Expected version 1 got 0
I20251028 03:00:04.299000 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet f62b981b9b1a44bb96bbc90f924c3b64 (table default.Table_3f85bca98f84492da9cc6b47c772334d [id=d4eaebd005bc42dd86d7125ef555546f]). Expected version 5 got 0
I20251028 03:00:04.299099 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet a38ef10c145a4d00a26265a9dee5c744 (table default.Table_8d72f9b8a87847d0a40f66c2bcdba651 [id=ba8b6a629ebc489abe7043890455765a]). Expected version 1 got 0
I20251028 03:00:04.300487 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ee2fd98744ff44028f36bb55e6eba51e
I20251028 03:00:04.302752 7966 catalog_manager.cc:6719] ReplaceTablet: tablet ee2fd98744ff44028f36bb55e6eba51e deleted and replaced by tablet adf43d587639446e8bb3959b490c931b
I20251028 03:00:04.299228 7964 catalog_manager.cc:5649] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } }
I20251028 03:00:04.305292 7964 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 4f90e3c3524e4498aa6c4bfc5304d44b (table default.Table_42f89d7fccbf4c738293e9796be235a0 [id=6f3e689a22ae4160ae7537d955c06a90]). Expected version 1 got 0
I20251028 03:00:04.305377 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet ee2fd98744ff44028f36bb55e6eba51e with delete_type TABLET_DATA_DELETED (replaced by tablet adf43d587639446e8bb3959b490c931b) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.307158 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet ee2fd98744ff44028f36bb55e6eba51e with delete_type TABLET_DATA_DELETED (replaced by tablet adf43d587639446e8bb3959b490c931b) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.307855 8550 ts_tablet_manager.cc:1916] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.308051 8546 tablet_bootstrap.cc:654] T 58268ac590f3437a942dcafd927a5496 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.308492 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet ee2fd98744ff44028f36bb55e6eba51e with delete_type TABLET_DATA_DELETED (replaced by tablet adf43d587639446e8bb3959b490c931b) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.309129 8551 tablet_replica.cc:333] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.309326 8551 raft_consensus.cc:2243] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.309437 8551 raft_consensus.cc:2272] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.309854 8551 ts_tablet_manager.cc:1916] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.310590 8550 ts_tablet_manager.cc:1929] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.310822 8550 log.cc:1199] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/d60fc654f6af44b290d46235082a4113
I20251028 03:00:04.311987 8550 ts_tablet_manager.cc:1950] T d60fc654f6af44b290d46235082a4113 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.312294 8551 ts_tablet_manager.cc:1929] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.312361 8551 log.cc:1199] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ee2fd98744ff44028f36bb55e6eba51e
I20251028 03:00:04.312566 8551 ts_tablet_manager.cc:1950] T ee2fd98744ff44028f36bb55e6eba51e P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.312695 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet d60fc654f6af44b290d46235082a4113 (table default.Table_a387569232e54415a0dd77e906ddcee4 [id=94895dadf0e34a19a454e8755c61c785]) successfully deleted
I20251028 03:00:04.312942 8550 tablet_replica.cc:333] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.313083 8550 raft_consensus.cc:2243] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.313181 8550 raft_consensus.cc:2272] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.313380 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet a52b2fe9282b486682a961fc2136557c
I20251028 03:00:04.313521 8550 ts_tablet_manager.cc:1916] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.313679 7960 catalog_manager.cc:6719] ReplaceTablet: tablet a52b2fe9282b486682a961fc2136557c deleted and replaced by tablet 4cfeaa22b8ed411a852fe81a1d998413
I20251028 03:00:04.314421 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet a52b2fe9282b486682a961fc2136557c with delete_type TABLET_DATA_DELETED (replaced by tablet 4cfeaa22b8ed411a852fe81a1d998413) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.316810 8550 ts_tablet_manager.cc:1929] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.316924 8550 log.cc:1199] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/4b787f8f024745b297b2ebaa456cbcbf
I20251028 03:00:04.317167 8550 ts_tablet_manager.cc:1950] T 4b787f8f024745b297b2ebaa456cbcbf P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.317864 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 4b787f8f024745b297b2ebaa456cbcbf (table default.Table_35aa9cb3afdc4a149341fa51b840258f [id=a05d3a9fbe8c4278bcab247dcd663653]) successfully deleted
I20251028 03:00:04.318204 8544 tablet_bootstrap.cc:492] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.318289 8544 ts_tablet_manager.cc:1403] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
W20251028 03:00:04.318403 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 16123c06b3d94acfa11145963bd31b6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16123c06b3d94acfa11145963bd31b6b already in progress: creating tablet
I20251028 03:00:04.318677 8550 tablet_replica.cc:333] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.318709 8544 raft_consensus.cc:359] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.318827 8544 raft_consensus.cc:385] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.318867 8544 raft_consensus.cc:740] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.319015 8550 raft_consensus.cc:2243] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.318987 8544 consensus_queue.cc:260] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.319407 8544 ts_tablet_manager.cc:1434] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.319597 8544 tablet_bootstrap.cc:492] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.320526 8544 tablet_bootstrap.cc:654] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.320698 7966 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e7689390da9341cc8efa3e3a38c15a47" } modify_external_catalogs: true
I20251028 03:00:04.320829 7966 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_e7689390da9341cc8efa3e3a38c15a47" } modify_external_catalogs: true
I20251028 03:00:04.320957 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ee2fd98744ff44028f36bb55e6eba51e (table default.Table_c90ec4690cc24b63ac21b51b090816e5 [id=e77c51294af5486f9c30cd2f2c2a4eec]) successfully deleted
I20251028 03:00:04.321137 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_513f1d270ec342698d12855862f98bb4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:04.321223 8552 tablet_replica.cc:333] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.321364 8552 raft_consensus.cc:2243] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.321460 8552 raft_consensus.cc:2272] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:04.321481 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_513f1d270ec342698d12855862f98bb4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.321760 7969 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6088be97ba4b4513b640257c2d69675a" } modify_external_catalogs: true
I20251028 03:00:04.321859 7969 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_6088be97ba4b4513b640257c2d69675a" } modify_external_catalogs: true
I20251028 03:00:04.322269 7968 master_service.cc:954] ReplaceTablet: received request to replace tablet d802f3cffe6a4e0cabbda0fb7fe495bc from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:04.322674 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet a52b2fe9282b486682a961fc2136557c with delete_type TABLET_DATA_DELETED (replaced by tablet 4cfeaa22b8ed411a852fe81a1d998413) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.321233 10465 raft_consensus.cc:493] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.322908 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet a52b2fe9282b486682a961fc2136557c with delete_type TABLET_DATA_DELETED (replaced by tablet 4cfeaa22b8ed411a852fe81a1d998413) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.320922 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_fbf2e50c6edf4fa9bd5db6c2c34d5a24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.323479 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fbf2e50c6edf4fa9bd5db6c2c34d5a24 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.322922 10465 raft_consensus.cc:515] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 03:00:04.325176 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet a52b2fe9282b486682a961fc2136557c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a52b2fe9282b486682a961fc2136557c already in progress: creating tablet
I20251028 03:00:04.325403 10465 leader_election.cc:290] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.325927 10187 tablet.cc:1722] T b4c612bc752847c2be0547cedfd665ae P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:04.326818 8552 ts_tablet_manager.cc:1916] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.328323 8550 raft_consensus.cc:2272] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.328733 8550 ts_tablet_manager.cc:1916] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.329247 8552 ts_tablet_manager.cc:1929] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.329313 8552 log.cc:1199] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/ee2fd98744ff44028f36bb55e6eba51e
I20251028 03:00:04.329468 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b0c593a3b96434db9ac42d260fc6997" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.329528 8552 ts_tablet_manager.cc:1950] T ee2fd98744ff44028f36bb55e6eba51e P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.329604 8419 raft_consensus.cc:2468] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:04.329888 10422 tablet.cc:1722] T b4c612bc752847c2be0547cedfd665ae P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:04.330224 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet ee2fd98744ff44028f36bb55e6eba51e (table default.Table_c90ec4690cc24b63ac21b51b090816e5 [id=e77c51294af5486f9c30cd2f2c2a4eec]) successfully deleted
I20251028 03:00:04.330790 8220 leader_election.cc:304] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:04.330981 8545 tablet_bootstrap.cc:492] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.331079 8545 ts_tablet_manager.cc:1403] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.000s
I20251028 03:00:04.331066 8553 raft_consensus.cc:2804] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.331118 8553 raft_consensus.cc:493] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.331156 8553 raft_consensus.cc:3060] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.331655 8545 raft_consensus.cc:359] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.331772 8545 raft_consensus.cc:385] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.331812 8545 raft_consensus.cc:740] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.331946 8545 consensus_queue.cc:260] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.332304 8545 ts_tablet_manager.cc:1434] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.332474 8545 tablet_bootstrap.cc:492] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.332841 8544 tablet_bootstrap.cc:492] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.332939 8544 ts_tablet_manager.cc:1403] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20251028 03:00:04.333453 8545 tablet_bootstrap.cc:654] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.333892 8544 raft_consensus.cc:359] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.333997 8544 raft_consensus.cc:385] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.334030 8544 raft_consensus.cc:740] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.334167 8544 consensus_queue.cc:260] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.334450 8544 ts_tablet_manager.cc:1434] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.334627 8544 tablet_bootstrap.cc:492] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.335918 8544 tablet_bootstrap.cc:654] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:04.336295 8218 leader_election.cc:343] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 3b0c593a3b96434db9ac42d260fc6997
I20251028 03:00:04.336994 8550 ts_tablet_manager.cc:1929] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.337045 10242 raft_consensus.cc:493] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.337093 8550 log.cc:1199] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/d5c3b02db24d4dc48260b42841844938
I20251028 03:00:04.337133 8546 tablet_bootstrap.cc:492] T 58268ac590f3437a942dcafd927a5496 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.337136 10242 raft_consensus.cc:515] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.337210 8546 ts_tablet_manager.cc:1403] T 58268ac590f3437a942dcafd927a5496 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.046s user 0.000s sys 0.002s
I20251028 03:00:04.337304 8550 ts_tablet_manager.cc:1950] T d5c3b02db24d4dc48260b42841844938 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.337550 10242 leader_election.cc:290] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.337960 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet d5c3b02db24d4dc48260b42841844938 (table default.Table_aa03ff4bf2694e7eabc52863451cba52 [id=3a8e945f16744123a8583ec0db730658]) successfully deleted
I20251028 03:00:04.338091 8546 raft_consensus.cc:359] T 58268ac590f3437a942dcafd927a5496 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.338213 8546 raft_consensus.cc:385] T 58268ac590f3437a942dcafd927a5496 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.338259 8546 raft_consensus.cc:740] T 58268ac590f3437a942dcafd927a5496 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.338268 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "463f1c3ac2d74791830431191601282c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.338394 8546 consensus_queue.cc:260] T 58268ac590f3437a942dcafd927a5496 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.338526 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "463f1c3ac2d74791830431191601282c" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.338678 8546 ts_tablet_manager.cc:1434] T 58268ac590f3437a942dcafd927a5496 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:04.338673 10423 tablet.cc:1722] T b4c612bc752847c2be0547cedfd665ae P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:04.338860 8546 tablet_bootstrap.cc:492] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 03:00:04.339918 8356 leader_election.cc:343] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.339952 8546 tablet_bootstrap.cc:654] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:04.340416 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet a52b2fe9282b486682a961fc2136557c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a52b2fe9282b486682a961fc2136557c already in progress: creating tablet
I20251028 03:00:04.340778 8550 tablet_replica.cc:333] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
W20251028 03:00:04.340773 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 32efb7b57678451f97abea5ea61da87a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32efb7b57678451f97abea5ea61da87a already in progress: creating tablet
I20251028 03:00:04.340961 8550 raft_consensus.cc:2243] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.340929 8553 raft_consensus.cc:515] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.341105 8550 raft_consensus.cc:2272] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.341367 8553 leader_election.cc:290] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.341502 8550 ts_tablet_manager.cc:1916] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.342605 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b0c593a3b96434db9ac42d260fc6997" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.342741 8420 raft_consensus.cc:3060] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.343474 7966 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 9b98c14370a341fd9939fceb536c8ed6
I20251028 03:00:04.343902 8420 raft_consensus.cc:2468] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:04.344348 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b98c14370a341fd9939fceb536c8ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.344512 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b98c14370a341fd9939fceb536c8ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.344558 8544 tablet_bootstrap.cc:492] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.344633 8544 ts_tablet_manager.cc:1403] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20251028 03:00:04.344638 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6f6f58ddaa6940248065a05b75996112
I20251028 03:00:04.344659 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 9b98c14370a341fd9939fceb536c8ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.344954 10242 raft_consensus.cc:493] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.345043 10242 raft_consensus.cc:515] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.345093 8544 raft_consensus.cc:359] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.345186 8544 raft_consensus.cc:385] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.345216 8544 raft_consensus.cc:740] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.345325 8544 consensus_queue.cc:260] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.345427 10242 leader_election.cc:290] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.345551 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f6f58ddaa6940248065a05b75996112 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.345582 8544 ts_tablet_manager.cc:1434] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 03:00:04.345602 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 9b98c14370a341fd9939fceb536c8ed6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b98c14370a341fd9939fceb536c8ed6 already in progress: creating tablet
I20251028 03:00:04.345783 8544 tablet_bootstrap.cc:492] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
W20251028 03:00:04.346309 8354 leader_election.cc:343] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.346372 8354 leader_election.cc:304] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.347383 8397 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f6f58ddaa6940248065a05b75996112 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.347385 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d5c3b02db24d4dc48260b42841844938" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.347473 8544 tablet_bootstrap.cc:654] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.347546 8261 tablet_service.cc:1552] Processing DeleteTablet for tablet 6f6f58ddaa6940248065a05b75996112 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.347716 8551 tablet_replica.cc:333] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.347872 8551 raft_consensus.cc:2243] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.347998 8551 raft_consensus.cc:2272] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.348140 8220 leader_election.cc:304] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
W20251028 03:00:04.348340 8356 leader_election.cc:343] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.348414 8551 ts_tablet_manager.cc:1916] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.348456 8545 tablet_bootstrap.cc:492] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.348532 8545 ts_tablet_manager.cc:1403] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 03:00:04.348599 10464 raft_consensus.cc:2804] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.348690 10464 raft_consensus.cc:697] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:04.348826 10464 consensus_queue.cc:237] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.349031 8545 raft_consensus.cc:359] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.349126 8545 raft_consensus.cc:385] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.349159 8545 raft_consensus.cc:740] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.349308 8545 consensus_queue.cc:260] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.349586 8545 ts_tablet_manager.cc:1434] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.349773 9190 raft_consensus.cc:2749] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.350827 8551 ts_tablet_manager.cc:1929] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:04.350898 8551 log.cc:1199] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/6f6f58ddaa6940248065a05b75996112
I20251028 03:00:04.351049 8545 tablet_bootstrap.cc:492] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.351155 8551 ts_tablet_manager.cc:1950] T 6f6f58ddaa6940248065a05b75996112 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.351915 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]) successfully deleted
I20251028 03:00:04.351967 8552 tablet_replica.cc:333] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.352111 8552 raft_consensus.cc:2243] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.352181 8545 tablet_bootstrap.cc:654] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.352310 8552 raft_consensus.cc:2272] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.352413 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ff74a0ebc6284f5f813ca17bd15504d4" } new_table_name: "default.Table_f90c49097fc4469985655460ded340b9" modify_external_catalogs: true
I20251028 03:00:04.352820 8552 ts_tablet_manager.cc:1916] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.353703 8550 ts_tablet_manager.cc:1929] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20251028 03:00:04.353840 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 9b98c14370a341fd9939fceb536c8ed6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b98c14370a341fd9939fceb536c8ed6 already in progress: creating tablet
I20251028 03:00:04.353915 8550 log.cc:1199] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/66d385494c7a4fb8a6501519925be7d4
I20251028 03:00:04.354272 8550 ts_tablet_manager.cc:1950] T 66d385494c7a4fb8a6501519925be7d4 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.355197 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 66d385494c7a4fb8a6501519925be7d4 (table default.Table_18a5674a59e34ce4b3bf54efce8a8a44 [id=20ee524d16f941159b5c44e85ed885f6]) successfully deleted
I20251028 03:00:04.356132 8552 ts_tablet_manager.cc:1929] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:04.356222 8550 tablet_replica.cc:333] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.356405 8550 raft_consensus.cc:2243] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.356520 10460 raft_consensus.cc:493] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.356611 10460 raft_consensus.cc:515] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.356775 8552 log.cc:1199] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/6f6f58ddaa6940248065a05b75996112
I20251028 03:00:04.357070 10460 leader_election.cc:290] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.357213 8552 ts_tablet_manager.cc:1950] T 6f6f58ddaa6940248065a05b75996112 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
W20251028 03:00:04.357650 8083 leader_election.cc:343] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: 28b81f0000c4407096feff773c29579d
I20251028 03:00:04.358089 8545 tablet_bootstrap.cc:492] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
W20251028 03:00:04.358321 8083 leader_election.cc:343] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 28b81f0000c4407096feff773c29579d
I20251028 03:00:04.358390 8083 leader_election.cc:304] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75
I20251028 03:00:04.358569 10460 raft_consensus.cc:2749] T 28b81f0000c4407096feff773c29579d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.358815 8545 ts_tablet_manager.cc:1403] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20251028 03:00:04.359442 8545 raft_consensus.cc:359] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.359611 8545 raft_consensus.cc:385] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.359689 8545 raft_consensus.cc:740] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
W20251028 03:00:04.360700 8354 leader_election.cc:343] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: d5c3b02db24d4dc48260b42841844938
I20251028 03:00:04.360776 8354 leader_election.cc:304] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.361052 8545 consensus_queue.cc:260] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:04.361116 8218 leader_election.cc:343] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 3b0c593a3b96434db9ac42d260fc6997
I20251028 03:00:04.361299 9190 raft_consensus.cc:2749] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.361755 8545 ts_tablet_manager.cc:1434] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 03:00:04.362008 8545 tablet_bootstrap.cc:492] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.362280 9190 raft_consensus.cc:493] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.362365 9190 raft_consensus.cc:515] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.362697 9190 leader_election.cc:290] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.363559 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3a5415a4246c47ca896751f2fd68af71" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.364064 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]) successfully deleted
W20251028 03:00:04.365154 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df88e048b2ee470a8af37a2aae6a8fc9 consensus state has no leader
W20251028 03:00:04.365630 8356 leader_election.cc:343] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.365738 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3a5415a4246c47ca896751f2fd68af71" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.365856 8146 raft_consensus.cc:2468] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:04.367668 8354 leader_election.cc:304] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.367921 10242 raft_consensus.cc:2804] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.367976 10242 raft_consensus.cc:493] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.368016 10242 raft_consensus.cc:3060] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.368702 8545 tablet_bootstrap.cc:654] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.369130 10242 raft_consensus.cc:515] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.369937 10242 leader_election.cc:290] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.369949 8544 tablet_bootstrap.cc:492] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.370043 8544 ts_tablet_manager.cc:1403] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20251028 03:00:04.370592 8544 raft_consensus.cc:359] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.370718 8544 raft_consensus.cc:385] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.370769 8544 raft_consensus.cc:740] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.370898 8544 consensus_queue.cc:260] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.371186 8544 ts_tablet_manager.cc:1434] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.371419 8544 tablet_bootstrap.cc:492] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.372471 8544 tablet_bootstrap.cc:654] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.373237 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3a5415a4246c47ca896751f2fd68af71" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:04.373239 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3a5415a4246c47ca896751f2fd68af71" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:04.373355 8146 raft_consensus.cc:3060] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:04.373624 8356 leader_election.cc:343] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.374330 8146 raft_consensus.cc:2468] T 3a5415a4246c47ca896751f2fd68af71 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:04.374819 8354 leader_election.cc:304] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.375263 9190 raft_consensus.cc:2804] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.375401 9190 raft_consensus.cc:697] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:04.375532 9190 consensus_queue.cc:237] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.376436 8546 tablet_bootstrap.cc:492] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.376536 8546 ts_tablet_manager.cc:1403] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20251028 03:00:04.377003 8546 raft_consensus.cc:359] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.377113 8546 raft_consensus.cc:385] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.377143 8546 raft_consensus.cc:740] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.377260 8546 consensus_queue.cc:260] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.377657 8546 ts_tablet_manager.cc:1434] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.378108 8546 tablet_bootstrap.cc:492] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.379242 8546 tablet_bootstrap.cc:654] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.379401 10465 raft_consensus.cc:493] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.379487 10465 raft_consensus.cc:515] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.379853 10465 leader_election.cc:290] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.380290 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a578008e735a4cc1a2f50dba09c16117" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.380429 8420 raft_consensus.cc:2468] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:04.380719 8220 leader_election.cc:304] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:04.380903 10465 raft_consensus.cc:2804] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.380977 10465 raft_consensus.cc:493] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.381017 10465 raft_consensus.cc:3060] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.382038 10465 raft_consensus.cc:515] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.382366 10465 leader_election.cc:290] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.382817 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a578008e735a4cc1a2f50dba09c16117" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.382874 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a578008e735a4cc1a2f50dba09c16117" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.382952 8420 raft_consensus.cc:3060] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.383013 8146 raft_consensus.cc:2468] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:04.356524 8550 raft_consensus.cc:2272] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.383800 8550 ts_tablet_manager.cc:1916] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.384240 8420 raft_consensus.cc:2468] T a578008e735a4cc1a2f50dba09c16117 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:04.384617 8220 leader_election.cc:304] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:04.384802 10465 raft_consensus.cc:2804] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.384882 10465 raft_consensus.cc:697] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:04.385021 10465 consensus_queue.cc:237] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.386300 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a578008e735a4cc1a2f50dba09c16117" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:04.386492 8146 raft_consensus.cc:3060] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.387018 8544 tablet_bootstrap.cc:492] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.387099 8544 ts_tablet_manager.cc:1403] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20251028 03:00:04.387773 8544 raft_consensus.cc:359] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.387894 8544 raft_consensus.cc:385] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.387931 8544 raft_consensus.cc:740] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.388600 8146 raft_consensus.cc:2468] T a578008e735a4cc1a2f50dba09c16117 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:04.388082 8544 consensus_queue.cc:260] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.389428 8544 ts_tablet_manager.cc:1434] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251028 03:00:04.389600 8544 tablet_bootstrap.cc:492] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.390662 8544 tablet_bootstrap.cc:654] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.394650 8550 ts_tablet_manager.cc:1929] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.394764 8550 log.cc:1199] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/e369dbabc2c04ad5ab17bbd4225127c1
I20251028 03:00:04.395020 8550 ts_tablet_manager.cc:1950] T e369dbabc2c04ad5ab17bbd4225127c1 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.395480 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet e369dbabc2c04ad5ab17bbd4225127c1 (table default.Table_621dda8a15654228acd735a8dca754ef [id=36f3b29b4d154e6caf4c3fb52bc82466]) successfully deleted
I20251028 03:00:04.395803 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_32faa291266d4d11b78e09816341a9a9" } new_table_name: "default.Table_7aae0892516e40f4a147692727a0e2f2" modify_external_catalogs: true
I20251028 03:00:04.395891 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_41baf3bfcbb445c7b7bdd02225bd7d0b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.396293 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41baf3bfcbb445c7b7bdd02225bd7d0b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.396327 8545 tablet_bootstrap.cc:492] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.396428 8545 ts_tablet_manager.cc:1403] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20251028 03:00:04.396495 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_56d9316f99f84ee6b2c7f5fec6b87e67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.396811 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_56d9316f99f84ee6b2c7f5fec6b87e67 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.397004 8545 raft_consensus.cc:359] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.397039 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f5ab03b8c69647239a7f03b068295ee2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:04.397118 8545 raft_consensus.cc:385] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.397161 8545 raft_consensus.cc:740] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
W20251028 03:00:04.397331 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f5ab03b8c69647239a7f03b068295ee2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.397323 8545 consensus_queue.cc:260] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.395828 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ec9c33981114406287553ea2e042b5e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:04.397645 8545 ts_tablet_manager.cc:1434] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20251028 03:00:04.397804 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec9c33981114406287553ea2e042b5e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.397862 8545 tablet_bootstrap.cc:492] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.399475 8545 tablet_bootstrap.cc:654] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.401084 8545 tablet_bootstrap.cc:492] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.401232 8545 ts_tablet_manager.cc:1403] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20251028 03:00:04.401744 10460 raft_consensus.cc:493] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.401733 8545 raft_consensus.cc:359] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.401849 8545 raft_consensus.cc:385] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.401885 8545 raft_consensus.cc:740] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.401851 10460 raft_consensus.cc:515] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.402022 8545 consensus_queue.cc:260] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.402271 10460 leader_election.cc:290] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.402779 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "583d6226a5f44d47a5ea4788c2b0d5ce" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:04.403180 8083 leader_election.cc:343] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.403527 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "583d6226a5f44d47a5ea4788c2b0d5ce" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.403640 8420 raft_consensus.cc:2468] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:04.403945 8083 leader_election.cc:304] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.404223 10460 raft_consensus.cc:2804] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.404277 10460 raft_consensus.cc:493] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.404316 10460 raft_consensus.cc:3060] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.405478 10460 raft_consensus.cc:515] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.405825 10460 leader_election.cc:290] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.406311 8545 ts_tablet_manager.cc:1434] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251028 03:00:04.409672 8545 tablet_bootstrap.cc:492] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.409749 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "583d6226a5f44d47a5ea4788c2b0d5ce" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:04.410326 8083 leader_election.cc:343] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.410866 8546 tablet_bootstrap.cc:492] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.411021 8546 ts_tablet_manager.cc:1403] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20251028 03:00:04.411142 8545 tablet_bootstrap.cc:654] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.411713 8546 raft_consensus.cc:359] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.411861 8546 raft_consensus.cc:385] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.411927 8546 raft_consensus.cc:740] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.412098 8546 consensus_queue.cc:260] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.414938 8546 ts_tablet_manager.cc:1434] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 03:00:04.415200 8546 tablet_bootstrap.cc:492] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 03:00:04.416285 7960 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0696349affc34573b08bcb84df29c201 consensus state has no leader
I20251028 03:00:04.416492 8550 tablet_replica.cc:333] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.416514 8546 tablet_bootstrap.cc:654] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.418715 8544 tablet_bootstrap.cc:492] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.418926 8544 ts_tablet_manager.cc:1403] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.002s
I20251028 03:00:04.419507 8544 raft_consensus.cc:359] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.419704 8544 raft_consensus.cc:385] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.419780 8544 raft_consensus.cc:740] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.419972 8544 consensus_queue.cc:260] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.420346 8544 ts_tablet_manager.cc:1434] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.420727 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "583d6226a5f44d47a5ea4788c2b0d5ce" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.420861 8420 raft_consensus.cc:3060] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.421916 8420 raft_consensus.cc:2468] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:04.423565 8544 tablet_bootstrap.cc:492] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.424855 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet f6818c3f2dfa48448bb2d9b38ed0dc19 (table default.Table_73967d33588f4f02ab5e0010826ec5f6 [id=24424b561e55419cbc0d32089a83e21a]). Expected version 1 got 0
I20251028 03:00:04.425060 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet f62b981b9b1a44bb96bbc90f924c3b64 (table default.Table_3f85bca98f84492da9cc6b47c772334d [id=d4eaebd005bc42dd86d7125ef555546f]). Expected version 5 got 0
I20251028 03:00:04.425123 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 583d6226a5f44d47a5ea4788c2b0d5ce (table default.Table_1e52c58d3e4042139159ba0e312a4089 [id=f5dcccbdb3e44a00a0169cff89823b0e]). Expected version 1 got 0
I20251028 03:00:04.425217 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 4f90e3c3524e4498aa6c4bfc5304d44b (table default.Table_42f89d7fccbf4c738293e9796be235a0 [id=6f3e689a22ae4160ae7537d955c06a90]). Expected version 1 got 0
I20251028 03:00:04.425311 7962 catalog_manager.cc:5649] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:04.425523 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 26a60624333d43f1b701036b7e9dfdb0 (table default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3]). Expected version 1 got 0
I20251028 03:00:04.425671 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet a38ef10c145a4d00a26265a9dee5c744 (table default.Table_8d72f9b8a87847d0a40f66c2bcdba651 [id=ba8b6a629ebc489abe7043890455765a]). Expected version 1 got 0
I20251028 03:00:04.425844 7963 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_8c67370d4f5c47888eda5c234f795044 [id=21585cd941ee4a3bb43066449e6d27e5] alter complete (version 1)
I20251028 03:00:04.426867 8261 tablet_service.cc:1505] Processing CreateTablet for tablet a91d524a12e54482acf1dbdcae4e6095 (DEFAULT_TABLE table=default.Table_829c6eb42be14ebbb7b97fec16a89ba6 [id=53070de7774c4adb9b5c55a0d737e904]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.427178 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91d524a12e54482acf1dbdcae4e6095. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.427251 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b0c593a3b96434db9ac42d260fc6997 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.427379 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet 8720ea8c68904d57a3fc5c0e94d338ca with delete_type TABLET_DATA_DELETED (replaced by tablet 30232298155542ecb41f5a1b19b16ffb) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.427779 8544 tablet_bootstrap.cc:654] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.427830 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 2b4d383e126e487390f6dbecc060be48 (DEFAULT_TABLE table=default.Table_e3764076fbd04d449a8b8249a205f2cf [id=e37829475829451e8d5c12e84600464f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.427932 8552 tablet_replica.cc:333] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.428079 8552 raft_consensus.cc:2243] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.428254 8552 pending_rounds.cc:70] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: Trying to abort 1 pending ops.
I20251028 03:00:04.428283 8552 pending_rounds.cc:77] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597176214228992 op_type: NO_OP noop_request { }
I20251028 03:00:04.428335 8552 raft_consensus.cc:2889] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:04.428359 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 3a04270a2fbd49c294c4a2f24672036d from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:04.428386 8552 raft_consensus.cc:2272] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.428076 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4d383e126e487390f6dbecc060be48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.429081 8552 ts_tablet_manager.cc:1916] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.429536 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet e369dbabc2c04ad5ab17bbd4225127c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.429809 8257 tablet_service.cc:1505] Processing CreateTablet for tablet d39c1ae69fb7428d9e271faf550df610 (DEFAULT_TABLE table=default.Table_ff79527661d5411d9933e17bf3c6675c [id=82b65083e16e4883ac5192eba96f0cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.430022 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d39c1ae69fb7428d9e271faf550df610. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.431161 8552 ts_tablet_manager.cc:1929] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.431223 8552 log.cc:1199] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3b0c593a3b96434db9ac42d260fc6997
I20251028 03:00:04.431334 8553 raft_consensus.cc:493] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.431428 8553 raft_consensus.cc:515] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.431545 8552 ts_tablet_manager.cc:1950] T 3b0c593a3b96434db9ac42d260fc6997 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.431811 8111 tablet_service.cc:1505] Processing CreateTablet for tablet 2b4d383e126e487390f6dbecc060be48 (DEFAULT_TABLE table=default.Table_e3764076fbd04d449a8b8249a205f2cf [id=e37829475829451e8d5c12e84600464f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.431877 8553 leader_election.cc:290] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.432040 8124 tablet_service.cc:1505] Processing CreateTablet for tablet a91d524a12e54482acf1dbdcae4e6095 (DEFAULT_TABLE table=default.Table_829c6eb42be14ebbb7b97fec16a89ba6 [id=53070de7774c4adb9b5c55a0d737e904]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.432272 8118 tablet_service.cc:1505] Processing CreateTablet for tablet d39c1ae69fb7428d9e271faf550df610 (DEFAULT_TABLE table=default.Table_ff79527661d5411d9933e17bf3c6675c [id=82b65083e16e4883ac5192eba96f0cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.432091 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4d383e126e487390f6dbecc060be48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.433231 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91d524a12e54482acf1dbdcae4e6095. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.434782 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 2fc2faa9c4f04fe3b931ce8a2808d738 (table default.Table_0d32d1fdb9bb4294a8b7054819d111e5 [id=eaebbf79fab746dfaa81407d3509306c]). Expected version 1 got 0
W20251028 03:00:04.434903 7955 catalog_manager.cc:4767] Async tablet task AlterTable RPC for tablet af93c341f684426fa8a847ee06d86d35 (table default.Table_1aebb42ceeac4980ac2abbbc5f435a56 [id=a87c19bc933b49b5bbbe15b571bc13cf], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet af93c341f684426fa8a847ee06d86d35 consensus state has no leader
I20251028 03:00:04.416654 8550 raft_consensus.cc:2243] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.435415 8550 raft_consensus.cc:2272] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.435411 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_db72e72bf1ce4c70a62bdf4b009366ac" } modify_external_catalogs: true
I20251028 03:00:04.435524 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_db72e72bf1ce4c70a62bdf4b009366ac" } modify_external_catalogs: true
I20251028 03:00:04.436167 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d39c1ae69fb7428d9e271faf550df610. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.436369 8124 tablet_service.cc:1505] Processing CreateTablet for tablet d167d7f94f57415a9e291a21d44394b6 (DEFAULT_TABLE table=default.Table_311b5d236e6248b5958556c596d0d986 [id=e33dcc161ed8415dbf464fbf6c9256bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.436604 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d167d7f94f57415a9e291a21d44394b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.436583 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_aee4903456c34845a3085d4f181b13ce" } modify_external_catalogs: true
I20251028 03:00:04.436671 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_aee4903456c34845a3085d4f181b13ce" } modify_external_catalogs: true
I20251028 03:00:04.437752 8257 tablet_service.cc:1505] Processing CreateTablet for tablet d167d7f94f57415a9e291a21d44394b6 (DEFAULT_TABLE table=default.Table_311b5d236e6248b5958556c596d0d986 [id=e33dcc161ed8415dbf464fbf6c9256bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.437789 7964 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_7cac0348a3bd4625b1daa54c5cd075b0" } new_table_name: "default.Table_ca1dc84a33ec493ba23cb3b38313db7d" modify_external_catalogs: true
I20251028 03:00:04.438015 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d167d7f94f57415a9e291a21d44394b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.440323 8550 ts_tablet_manager.cc:1916] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:04.440579 8083 consensus_peers.cc:597] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee2fd98744ff44028f36bb55e6eba51e. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.441210 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_dddd9c465f0b4e4cad048c3b984cbab3" } modify_external_catalogs: true
I20251028 03:00:04.441322 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_dddd9c465f0b4e4cad048c3b984cbab3" } modify_external_catalogs: true
I20251028 03:00:04.442473 8550 ts_tablet_manager.cc:1929] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.442543 8550 log.cc:1199] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/bedb4fd3b46f4453b194cfaea0c3c701
I20251028 03:00:04.442601 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3b0c593a3b96434db9ac42d260fc6997 (table default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]) successfully deleted
I20251028 03:00:04.442763 8550 ts_tablet_manager.cc:1950] T bedb4fd3b46f4453b194cfaea0c3c701 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.442881 10187 tablet.cc:1722] T e3b33c4d189247a89878b6a8e1fcbac0 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:04.443346 8550 tablet_replica.cc:333] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.443444 10423 tablet.cc:1722] T e3b33c4d189247a89878b6a8e1fcbac0 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:04.443483 8550 raft_consensus.cc:2243] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.443589 8550 raft_consensus.cc:2272] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.443979 8550 ts_tablet_manager.cc:1916] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.441082 8552 tablet_replica.cc:333] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.445734 8552 raft_consensus.cc:2243] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.445892 8550 ts_tablet_manager.cc:1929] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.445969 8550 log.cc:1199] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/a4bcdc8a0c554b7cae81d7993e4f2861
I20251028 03:00:04.446174 8550 ts_tablet_manager.cc:1950] T a4bcdc8a0c554b7cae81d7993e4f2861 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.446805 8550 tablet_replica.cc:333] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.446935 8550 raft_consensus.cc:2243] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.447106 8550 pending_rounds.cc:70] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 03:00:04.447141 8550 pending_rounds.cc:77] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597174173241344 op_type: NO_OP noop_request { }
I20251028 03:00:04.447193 8550 raft_consensus.cc:2889] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:04.447244 8550 raft_consensus.cc:2272] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.448966 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6455fca4d6684904a255751ce63258f0
I20251028 03:00:04.449214 8552 raft_consensus.cc:2272] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.449316 8553 raft_consensus.cc:493] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.449417 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet b2307c69e7ee47cbac31c54be957f5d4
I20251028 03:00:04.449409 8553 raft_consensus.cc:515] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.449815 8553 leader_election.cc:290] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.450059 8552 ts_tablet_manager.cc:1916] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:04.450296 7964 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 11f8b9bffa3048bdae44d3eaf25cf1f1 consensus state has no leader
W20251028 03:00:04.450606 8220 leader_election.cc:343] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: e369dbabc2c04ad5ab17bbd4225127c1
I20251028 03:00:04.453369 8546 tablet_bootstrap.cc:492] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.453467 8546 ts_tablet_manager.cc:1403] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20251028 03:00:04.453991 8546 raft_consensus.cc:359] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.434927 7965 catalog_manager.cc:5649] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e4cf694f71438298dd32a346abcf75 (127.7.154.131). New cstate: current_term: 1 leader_uuid: "d7e4cf694f71438298dd32a346abcf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:04.454100 8546 raft_consensus.cc:385] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.454135 8546 raft_consensus.cc:740] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.454178 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet 3a5415a4246c47ca896751f2fd68af71 (table default.Table_4bafc1e32c454d84b181dadf3727efcc [id=077d81b3977448b2ae212436d5014339]). Expected version 1 got 0
I20251028 03:00:04.454257 8546 consensus_queue.cc:260] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.454459 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 455fdd18fb6a4c15afd09ebeb30cb756 (DEFAULT_TABLE table=default.Table_0bd1a82a268f4a798071fba7b155d35c [id=5c85ed15e4664b088e48e6414ed225b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.454579 8546 ts_tablet_manager.cc:1434] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.454741 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 455fdd18fb6a4c15afd09ebeb30cb756. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.454752 8546 tablet_bootstrap.cc:492] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.455154 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet bedb4fd3b46f4453b194cfaea0c3c701 (table default.Table_c149f5a7a9164664a7a699ee42b93697 [id=ec9d6280f40442a38e13372a76e9042f]) successfully deleted
I20251028 03:00:04.455852 8546 tablet_bootstrap.cc:654] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.456954 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet 6455fca4d6684904a255751ce63258f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.457149 8115 tablet_service.cc:1505] Processing CreateTablet for tablet 04412f0a327243e895a555334d680c13 (DEFAULT_TABLE table=default.Table_974b645c67e142348f96878a0bc3132a [id=27af5cf17b5a45cfa5094d7dd4dcae63]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.457231 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 9cdb02fb1b994c1bbfb18543b9776b4c (DEFAULT_TABLE table=default.Table_98c5f51b251e48de9edb59891dbb28d6 [id=08a58524839942a6ae33cbc2a28f057d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.454718 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet b2307c69e7ee47cbac31c54be957f5d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.457371 8124 tablet_service.cc:1505] Processing CreateTablet for tablet ece8bffc9f8347e6bf87808d3a61fb7d (DEFAULT_TABLE table=default.Table_71f1dfc334c447899b2d951d85d4547a [id=b76fa9078add4245a655c9cca1d63cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.457485 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 0b6577bf30404cd6b27e0b9f17019137 (DEFAULT_TABLE table=default.Table_4d022cc0789d48a485d60399613331f6 [id=a28a4480ffea4de8b6394ff17a02679d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.457940 8111 tablet_service.cc:1505] Processing CreateTablet for tablet edf3884d16ec4706b75051f72d96a3b2 (DEFAULT_TABLE table=default.Table_19049cf1e5f2419fbff05d1df500a3ae [id=091f678d62af4a0792dfccedb3db996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.458575 8265 tablet_service.cc:1552] Processing DeleteTablet for tablet 6455fca4d6684904a255751ce63258f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.458614 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 0b6577bf30404cd6b27e0b9f17019137 (DEFAULT_TABLE table=default.Table_4d022cc0789d48a485d60399613331f6 [id=a28a4480ffea4de8b6394ff17a02679d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.458992 8257 tablet_service.cc:1505] Processing CreateTablet for tablet ece8bffc9f8347e6bf87808d3a61fb7d (DEFAULT_TABLE table=default.Table_71f1dfc334c447899b2d951d85d4547a [id=b76fa9078add4245a655c9cca1d63cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.459122 8261 tablet_service.cc:1505] Processing CreateTablet for tablet edf3884d16ec4706b75051f72d96a3b2 (DEFAULT_TABLE table=default.Table_19049cf1e5f2419fbff05d1df500a3ae [id=091f678d62af4a0792dfccedb3db996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.459618 8256 tablet_service.cc:1505] Processing CreateTablet for tablet 455fdd18fb6a4c15afd09ebeb30cb756 (DEFAULT_TABLE table=default.Table_0bd1a82a268f4a798071fba7b155d35c [id=5c85ed15e4664b088e48e6414ed225b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.459676 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b6577bf30404cd6b27e0b9f17019137. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.460032 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet a4bcdc8a0c554b7cae81d7993e4f2861 (table default.Table_806d85836a4c44a986095f5108a93d7f [id=c098c2727dd147e69471353bd9a741fc]) successfully deleted
W20251028 03:00:04.460217 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 67f3b694dcc5465ca2c8c5535e7efea6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 67f3b694dcc5465ca2c8c5535e7efea6 already in progress: creating tablet
I20251028 03:00:04.460558 8553 raft_consensus.cc:493] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.460634 8553 raft_consensus.cc:515] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.461009 8553 leader_election.cc:290] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:04.461733 8220 leader_election.cc:343] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 3435803c6167447a8b3afba6c0f3511b
W20251028 03:00:04.462432 8218 leader_election.cc:343] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: e369dbabc2c04ad5ab17bbd4225127c1
I20251028 03:00:04.462500 8218 leader_election.cc:304] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
W20251028 03:00:04.462677 8218 leader_election.cc:343] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 3435803c6167447a8b3afba6c0f3511b
I20251028 03:00:04.462725 8218 leader_election.cc:304] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.462970 10464 raft_consensus.cc:2749] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.463089 10464 raft_consensus.cc:2749] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.463137 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a60624333d43f1b701036b7e9dfdb0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.463315 8115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04412f0a327243e895a555334d680c13. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.463501 8122 tablet_service.cc:1505] Processing CreateTablet for tablet af83fc5f22944800b924c2ee8c2a3903 (DEFAULT_TABLE table=default.Table_9e3203e9487048d881979b269f95368f [id=5550a66b30b44a46b30253d2a192b44f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.463732 8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af83fc5f22944800b924c2ee8c2a3903. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.463920 8113 tablet_service.cc:1505] Processing CreateTablet for tablet c3ea9b2e9deb4a0b932a8a65ba321f3a (DEFAULT_TABLE table=default.Table_67078ec90ffe4bb7bbf75491a56d0b47 [id=836d42f705134a299486763a7a743f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.464123 8113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3ea9b2e9deb4a0b932a8a65ba321f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.465377 8116 tablet_service.cc:1552] Processing DeleteTablet for tablet 2fec0adb79d04469b90c1ea2a5e9ba5b with delete_type TABLET_DATA_DELETED (replaced by tablet f898e9072d7d4697aee6a981cf6188b4) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.465490 8112 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dd563e13e624d4bb88950b46d480876 with delete_type TABLET_DATA_DELETED (replaced by tablet 2fc2faa9c4f04fe3b931ce8a2808d738) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.465575 8126 tablet_service.cc:1552] Processing DeleteTablet for tablet e9ad4ae17665483584ce2d638d1639fb with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.465660 8128 tablet_service.cc:1552] Processing DeleteTablet for tablet 46ae536d269d4cbdbcee44a26d25bd67 with delete_type TABLET_DATA_DELETED (replaced by tablet ada2c7c883014aa597ad8dbe14bf51b3) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.465705 8117 tablet_service.cc:1552] Processing DeleteTablet for tablet ee1d686a0e4c47f6b214224047a55747 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.465878 8125 tablet_service.cc:1505] Processing CreateTablet for tablet 2278777b0ffe446cb1090962c27819ed (DEFAULT_TABLE table=default.Table_c725c121a0924531afb9b464c627b680 [id=adf5d206851d4c1c966d0f6ab90ecb1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.466107 8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2278777b0ffe446cb1090962c27819ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.466733 8265 tablet_service.cc:1505] Processing CreateTablet for tablet 9cdb02fb1b994c1bbfb18543b9776b4c (DEFAULT_TABLE table=default.Table_98c5f51b251e48de9edb59891dbb28d6 [id=08a58524839942a6ae33cbc2a28f057d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.467392 8256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 455fdd18fb6a4c15afd09ebeb30cb756. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.468268 8120 tablet_service.cc:1505] Processing CreateTablet for tablet 64ff84e80f4647d9a7c6eea42d11e56e (DEFAULT_TABLE table=default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.468480 8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ff84e80f4647d9a7c6eea42d11e56e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.469041 8249 tablet_service.cc:1505] Processing CreateTablet for tablet af83fc5f22944800b924c2ee8c2a3903 (DEFAULT_TABLE table=default.Table_9e3203e9487048d881979b269f95368f [id=5550a66b30b44a46b30253d2a192b44f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.469092 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet b2307c69e7ee47cbac31c54be957f5d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.469048 8247 tablet_service.cc:1505] Processing CreateTablet for tablet 04412f0a327243e895a555334d680c13 (DEFAULT_TABLE table=default.Table_974b645c67e142348f96878a0bc3132a [id=27af5cf17b5a45cfa5094d7dd4dcae63]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.470182 8249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af83fc5f22944800b924c2ee8c2a3903. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.470551 8261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf3884d16ec4706b75051f72d96a3b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.470573 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cdb02fb1b994c1bbfb18543b9776b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.470733 8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece8bffc9f8347e6bf87808d3a61fb7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.470863 8247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04412f0a327243e895a555334d680c13. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.471392 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cdb02fb1b994c1bbfb18543b9776b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.472105 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 2278777b0ffe446cb1090962c27819ed (DEFAULT_TABLE table=default.Table_c725c121a0924531afb9b464c627b680 [id=adf5d206851d4c1c966d0f6ab90ecb1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.472182 8255 tablet_service.cc:1505] Processing CreateTablet for tablet c3ea9b2e9deb4a0b932a8a65ba321f3a (DEFAULT_TABLE table=default.Table_67078ec90ffe4bb7bbf75491a56d0b47 [id=836d42f705134a299486763a7a743f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.472535 8252 tablet_service.cc:1505] Processing CreateTablet for tablet 31e339967daa42a490678deea80c1c18 (DEFAULT_TABLE table=default.Table_221b934925544c1180e7a1ee9c651c8e [id=3698a826ff0847faa51c0af1797d4761]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.472918 8260 tablet_service.cc:1505] Processing CreateTablet for tablet 64ff84e80f4647d9a7c6eea42d11e56e (DEFAULT_TABLE table=default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.473088 8118 tablet_service.cc:1505] Processing CreateTablet for tablet 31e339967daa42a490678deea80c1c18 (DEFAULT_TABLE table=default.Table_221b934925544c1180e7a1ee9c651c8e [id=3698a826ff0847faa51c0af1797d4761]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.473317 8118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e339967daa42a490678deea80c1c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.473476 8257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece8bffc9f8347e6bf87808d3a61fb7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.473510 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b6577bf30404cd6b27e0b9f17019137. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.476179 8111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf3884d16ec4706b75051f72d96a3b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.479873 8552 ts_tablet_manager.cc:1929] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.480052 8552 log.cc:1199] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/8720ea8c68904d57a3fc5c0e94d338ca
I20251028 03:00:04.480343 8552 ts_tablet_manager.cc:1950] T 8720ea8c68904d57a3fc5c0e94d338ca P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.481045 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2278777b0ffe446cb1090962c27819ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.481274 8260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ff84e80f4647d9a7c6eea42d11e56e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.481331 8552 tablet_replica.cc:333] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.481490 8552 raft_consensus.cc:2243] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.481602 8552 raft_consensus.cc:2272] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.481734 8255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3ea9b2e9deb4a0b932a8a65ba321f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.482095 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 8720ea8c68904d57a3fc5c0e94d338ca (table default.Table_79ced5936022425c91473064b87986d3 [id=375c6744f94644199d8d6f08f21ea021]) successfully deleted
I20251028 03:00:04.482141 8552 ts_tablet_manager.cc:1916] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.482151 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7ec99736e6604b9db63106eeb7272ea6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.482606 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ec99736e6604b9db63106eeb7272ea6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.482851 8546 tablet_bootstrap.cc:492] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.482923 8546 ts_tablet_manager.cc:1403] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.000s
I20251028 03:00:04.483446 8546 raft_consensus.cc:359] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.483548 8546 raft_consensus.cc:385] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.483502 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_33466204b62d4bdaa0ebe762e86b9bf2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:04.483578 8546 raft_consensus.cc:740] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.483690 8546 consensus_queue.cc:260] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:04.483886 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33466204b62d4bdaa0ebe762e86b9bf2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.483964 8546 ts_tablet_manager.cc:1434] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.483971 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet ac782a9f0b564667a7b20f1e328f2572
I20251028 03:00:04.484185 8546 tablet_bootstrap.cc:492] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.484258 8550 ts_tablet_manager.cc:1916] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.485183 8546 tablet_bootstrap.cc:654] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.486829 8247 tablet_service.cc:1552] Processing DeleteTablet for tablet ac782a9f0b564667a7b20f1e328f2572 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.487053 8252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e339967daa42a490678deea80c1c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.490574 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_ad4b160642444bafbfb3e9fc57fb7479"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.490940 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad4b160642444bafbfb3e9fc57fb7479 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.491227 8258 tablet_service.cc:1505] Processing CreateTablet for tablet 217ee23af795470d82c3e3e059a1b8a5 (DEFAULT_TABLE table=default.Table_1583dbc0fd384dea83db72bc0b3c8354 [id=535386602399465da8e806ba80d33776]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.491501 8258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 217ee23af795470d82c3e3e059a1b8a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.491469 8550 ts_tablet_manager.cc:1929] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.491461 8265 tablet_service.cc:1505] Processing CreateTablet for tablet 5e0d161e9a3b42d18f8713063f174a8d (DEFAULT_TABLE table=default.Table_dc630fed688f43898c0beceb9e1c6b9a [id=7a1691d8767e499ab800c7773853eb0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.491559 8550 log.cc:1199] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ee2fd98744ff44028f36bb55e6eba51e
I20251028 03:00:04.491673 8265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e0d161e9a3b42d18f8713063f174a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.491863 8550 ts_tablet_manager.cc:1950] T ee2fd98744ff44028f36bb55e6eba51e P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.494590 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet ee2fd98744ff44028f36bb55e6eba51e (table default.Table_c90ec4690cc24b63ac21b51b090816e5 [id=e77c51294af5486f9c30cd2f2c2a4eec]) successfully deleted
I20251028 03:00:04.494572 8127 tablet_service.cc:1505] Processing CreateTablet for tablet 217ee23af795470d82c3e3e059a1b8a5 (DEFAULT_TABLE table=default.Table_1583dbc0fd384dea83db72bc0b3c8354 [id=535386602399465da8e806ba80d33776]), partition=RANGE (key) PARTITION UNBOUNDED
W20251028 03:00:04.494730 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet a52b2fe9282b486682a961fc2136557c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a52b2fe9282b486682a961fc2136557c already in progress: creating tablet
I20251028 03:00:04.494773 8121 tablet_service.cc:1505] Processing CreateTablet for tablet 5e0d161e9a3b42d18f8713063f174a8d (DEFAULT_TABLE table=default.Table_dc630fed688f43898c0beceb9e1c6b9a [id=7a1691d8767e499ab800c7773853eb0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.494844 8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 217ee23af795470d82c3e3e059a1b8a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.494858 8550 tablet_replica.cc:333] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
W20251028 03:00:04.494875 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 9b98c14370a341fd9939fceb536c8ed6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b98c14370a341fd9939fceb536c8ed6 already in progress: creating tablet
I20251028 03:00:04.494997 8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e0d161e9a3b42d18f8713063f174a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.495016 8550 raft_consensus.cc:2243] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.495136 8550 raft_consensus.cc:2272] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.495556 8550 ts_tablet_manager.cc:1916] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.496662 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 26a60624333d43f1b701036b7e9dfdb0 (table default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3]). Expected version 1 got 0
I20251028 03:00:04.496785 7966 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 58268ac590f3437a942dcafd927a5496 (table default.Table_0b7b98e5884a48de833ab1fbbad97f08 [id=fdb289a561dd4c6497173ef43e2e9ccd]). Expected version 1 got 0
I20251028 03:00:04.423697 8545 tablet_bootstrap.cc:492] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.496905 7966 catalog_manager.cc:5649] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda reported cstate change: term changed from 0 to 1, leader changed from <none> to ccbdf478f18948d19de96574a685bfda (127.7.154.130). New cstate: current_term: 1 leader_uuid: "ccbdf478f18948d19de96574a685bfda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: HEALTHY } } }
I20251028 03:00:04.497088 8545 ts_tablet_manager.cc:1403] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.087s user 0.000s sys 0.003s
I20251028 03:00:04.497637 8545 raft_consensus.cc:359] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.497743 8545 raft_consensus.cc:385] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.497732 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet ac782a9f0b564667a7b20f1e328f2572 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.497787 8545 raft_consensus.cc:740] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.497896 8545 consensus_queue.cc:260] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.498241 8545 ts_tablet_manager.cc:1434] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:04.498443 8545 tablet_bootstrap.cc:492] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.498461 10469 raft_consensus.cc:493] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.498539 10469 raft_consensus.cc:515] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.498950 10469 leader_election.cc:290] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.499657 8083 leader_election.cc:304] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.500522 10469 raft_consensus.cc:2804] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.500608 10469 raft_consensus.cc:697] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:04.500732 10469 consensus_queue.cc:237] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.501519 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc2faa9c4f04fe3b931ce8a2808d738" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.501633 8421 raft_consensus.cc:2468] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:04.505244 8552 ts_tablet_manager.cc:1929] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.505333 8552 log.cc:1199] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/e369dbabc2c04ad5ab17bbd4225127c1
I20251028 03:00:04.505568 8552 ts_tablet_manager.cc:1950] T e369dbabc2c04ad5ab17bbd4225127c1 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.506860 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet e369dbabc2c04ad5ab17bbd4225127c1 (table default.Table_621dda8a15654228acd735a8dca754ef [id=36f3b29b4d154e6caf4c3fb52bc82466]) successfully deleted
W20251028 03:00:04.507118 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 6455fca4d6684904a255751ce63258f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6455fca4d6684904a255751ce63258f0 already in progress: creating tablet
I20251028 03:00:04.507222 8552 tablet_replica.cc:333] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 03:00:04.507237 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet b2307c69e7ee47cbac31c54be957f5d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2307c69e7ee47cbac31c54be957f5d4 already in progress: creating tablet
I20251028 03:00:04.507380 8552 raft_consensus.cc:2243] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.507486 8552 raft_consensus.cc:2272] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.507432 7960 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_c7534bd73076465c96d28ee8be7d5800" } new_table_name: "default.Table_a058dd240b5e45d68a76c604a5fb0ef6" modify_external_catalogs: true
I20251028 03:00:04.427579 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 2b4d383e126e487390f6dbecc060be48 (DEFAULT_TABLE table=default.Table_e3764076fbd04d449a8b8249a205f2cf [id=e37829475829451e8d5c12e84600464f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.507846 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b4d383e126e487390f6dbecc060be48. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.507956 8552 ts_tablet_manager.cc:1916] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.508299 7963 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_55aa729834954e5cadb191355a8a754e" } new_table_name: "default.Table_c2346703e13c4721bf2a911b902e94ee" modify_external_catalogs: true
I20251028 03:00:04.509425 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_7e6212b144d24c1ba6397f12a408bea5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.509795 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e6212b144d24c1ba6397f12a408bea5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.510475 8387 tablet_service.cc:1505] Processing CreateTablet for tablet 64ff84e80f4647d9a7c6eea42d11e56e (DEFAULT_TABLE table=default.Table_fdb7d4a7056c43e59a1dbb37a85b7ab7 [id=06998dad5ba54b30857bdd593aaf94a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.510727 8387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ff84e80f4647d9a7c6eea42d11e56e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.427802 8396 tablet_service.cc:1505] Processing CreateTablet for tablet d39c1ae69fb7428d9e271faf550df610 (DEFAULT_TABLE table=default.Table_ff79527661d5411d9933e17bf3c6675c [id=82b65083e16e4883ac5192eba96f0cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.511523 8396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d39c1ae69fb7428d9e271faf550df610. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.512212 8552 ts_tablet_manager.cc:1929] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:04.431541 8397 tablet_service.cc:1505] Processing CreateTablet for tablet a91d524a12e54482acf1dbdcae4e6095 (DEFAULT_TABLE table=default.Table_829c6eb42be14ebbb7b97fec16a89ba6 [id=53070de7774c4adb9b5c55a0d737e904]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.512276 8552 log.cc:1199] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/ac782a9f0b564667a7b20f1e328f2572
I20251028 03:00:04.512475 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91d524a12e54482acf1dbdcae4e6095. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.512578 8552 ts_tablet_manager.cc:1950] T ac782a9f0b564667a7b20f1e328f2572 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.513819 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet ac782a9f0b564667a7b20f1e328f2572 (table default.Table_dddd9c465f0b4e4cad048c3b984cbab3 [id=11719def996f48e68ac7d4ee2dbf66c5]) successfully deleted
I20251028 03:00:04.514876 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet f6818c3f2dfa48448bb2d9b38ed0dc19 (table default.Table_73967d33588f4f02ab5e0010826ec5f6 [id=24424b561e55419cbc0d32089a83e21a]). Expected version 1 got 0
I20251028 03:00:04.514951 8397 tablet_service.cc:1505] Processing CreateTablet for tablet c3ea9b2e9deb4a0b932a8a65ba321f3a (DEFAULT_TABLE table=default.Table_67078ec90ffe4bb7bbf75491a56d0b47 [id=836d42f705134a299486763a7a743f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.515092 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 3a5415a4246c47ca896751f2fd68af71 (table default.Table_4bafc1e32c454d84b181dadf3727efcc [id=077d81b3977448b2ae212436d5014339]). Expected version 1 got 0
I20251028 03:00:04.515162 8397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3ea9b2e9deb4a0b932a8a65ba321f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.515190 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 26a60624333d43f1b701036b7e9dfdb0 (table default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3]). Expected version 1 got 0
I20251028 03:00:04.432610 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dd563e13e624d4bb88950b46d480876 with delete_type TABLET_DATA_DELETED (replaced by tablet 2fc2faa9c4f04fe3b931ce8a2808d738) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.433058 8392 tablet_service.cc:1552] Processing DeleteTablet for tablet d5c3b02db24d4dc48260b42841844938 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:01 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.433144 8385 tablet_service.cc:1552] Processing DeleteTablet for tablet d60fc654f6af44b290d46235082a4113 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d42a5987fa540ce86afdbbe1922896e) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.515477 8386 tablet_service.cc:1552] Processing DeleteTablet for tablet ac782a9f0b564667a7b20f1e328f2572 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.515609 8385 tablet_service.cc:1505] Processing CreateTablet for tablet 5e0d161e9a3b42d18f8713063f174a8d (DEFAULT_TABLE table=default.Table_dc630fed688f43898c0beceb9e1c6b9a [id=7a1691d8767e499ab800c7773853eb0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.515664 8386 tablet_service.cc:1505] Processing CreateTablet for tablet 217ee23af795470d82c3e3e059a1b8a5 (DEFAULT_TABLE table=default.Table_1583dbc0fd384dea83db72bc0b3c8354 [id=535386602399465da8e806ba80d33776]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.433355 8400 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b787f8f024745b297b2ebaa456cbcbf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.515828 8551 tablet_replica.cc:333] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.433432 8401 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b0c593a3b96434db9ac42d260fc6997 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.436028 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "26a60624333d43f1b701036b7e9dfdb0" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.438383 8398 tablet_service.cc:1505] Processing CreateTablet for tablet d167d7f94f57415a9e291a21d44394b6 (DEFAULT_TABLE table=default.Table_311b5d236e6248b5958556c596d0d986 [id=e33dcc161ed8415dbf464fbf6c9256bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.515971 8551 raft_consensus.cc:2243] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.516000 8419 raft_consensus.cc:2393] T 26a60624333d43f1b701036b7e9dfdb0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in this term.
I20251028 03:00:04.516064 8551 raft_consensus.cc:2272] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.515834 8385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e0d161e9a3b42d18f8713063f174a8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.458487 8389 tablet_service.cc:1505] Processing CreateTablet for tablet 455fdd18fb6a4c15afd09ebeb30cb756 (DEFAULT_TABLE table=default.Table_0bd1a82a268f4a798071fba7b155d35c [id=5c85ed15e4664b088e48e6414ed225b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.516440 8551 ts_tablet_manager.cc:1916] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.516480 8389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 455fdd18fb6a4c15afd09ebeb30cb756. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.516623 8386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 217ee23af795470d82c3e3e059a1b8a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.458707 8390 tablet_service.cc:1505] Processing CreateTablet for tablet ece8bffc9f8347e6bf87808d3a61fb7d (DEFAULT_TABLE table=default.Table_71f1dfc334c447899b2d951d85d4547a [id=b76fa9078add4245a655c9cca1d63cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.458709 8384 tablet_service.cc:1552] Processing DeleteTablet for tablet 6455fca4d6684904a255751ce63258f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.458920 8388 tablet_service.cc:1505] Processing CreateTablet for tablet 0b6577bf30404cd6b27e0b9f17019137 (DEFAULT_TABLE table=default.Table_4d022cc0789d48a485d60399613331f6 [id=a28a4480ffea4de8b6394ff17a02679d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.459419 8391 tablet_service.cc:1505] Processing CreateTablet for tablet edf3884d16ec4706b75051f72d96a3b2 (DEFAULT_TABLE table=default.Table_19049cf1e5f2419fbff05d1df500a3ae [id=091f678d62af4a0792dfccedb3db996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.466352 8393 tablet_service.cc:1552] Processing DeleteTablet for tablet b2307c69e7ee47cbac31c54be957f5d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
W20251028 03:00:04.517177 8354 consensus_peers.cc:597] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 -> Peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Couldn't send request to peer fbd2afb4894c4e7a8520cc09f8593aa0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b787f8f024745b297b2ebaa456cbcbf. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.466614 8399 tablet_service.cc:1505] Processing CreateTablet for tablet 9cdb02fb1b994c1bbfb18543b9776b4c (DEFAULT_TABLE table=default.Table_98c5f51b251e48de9edb59891dbb28d6 [id=08a58524839942a6ae33cbc2a28f057d]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.469818 8395 tablet_service.cc:1505] Processing CreateTablet for tablet 04412f0a327243e895a555334d680c13 (DEFAULT_TABLE table=default.Table_974b645c67e142348f96878a0bc3132a [id=27af5cf17b5a45cfa5094d7dd4dcae63]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.469859 8394 tablet_service.cc:1505] Processing CreateTablet for tablet af83fc5f22944800b924c2ee8c2a3903 (DEFAULT_TABLE table=default.Table_9e3203e9487048d881979b269f95368f [id=5550a66b30b44a46b30253d2a192b44f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.517573 8083 leader_election.cc:304] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:04.473173 8383 tablet_service.cc:1505] Processing CreateTablet for tablet 31e339967daa42a490678deea80c1c18 (DEFAULT_TABLE table=default.Table_221b934925544c1180e7a1ee9c651c8e [id=3698a826ff0847faa51c0af1797d4761]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.473302 8382 tablet_service.cc:1505] Processing CreateTablet for tablet 2278777b0ffe446cb1090962c27819ed (DEFAULT_TABLE table=default.Table_c725c121a0924531afb9b464c627b680 [id=adf5d206851d4c1c966d0f6ab90ecb1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251028 03:00:04.473344 9190 raft_consensus.cc:493] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.518042 9190 raft_consensus.cc:515] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.518395 9190 leader_election.cc:290] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.518654 8220 leader_election.cc:304] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.518940 10469 raft_consensus.cc:2804] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.519011 10469 raft_consensus.cc:493] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.519047 10469 raft_consensus.cc:3060] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.519223 8553 raft_consensus.cc:3060] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.520545 10469 raft_consensus.cc:515] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.520900 10469 leader_election.cc:290] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.521071 8398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d167d7f94f57415a9e291a21d44394b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.522468 8544 tablet_bootstrap.cc:492] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.522557 8544 ts_tablet_manager.cc:1403] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.099s user 0.002s sys 0.000s
I20251028 03:00:04.523093 8544 raft_consensus.cc:359] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.523187 8544 raft_consensus.cc:385] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.523216 8544 raft_consensus.cc:740] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.523334 8551 ts_tablet_manager.cc:1929] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.523402 8551 log.cc:1199] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/6dd563e13e624d4bb88950b46d480876
I20251028 03:00:04.523372 8544 consensus_queue.cc:260] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.523620 8551 ts_tablet_manager.cc:1950] T 6dd563e13e624d4bb88950b46d480876 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.523695 8544 ts_tablet_manager.cc:1434] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.523797 10422 tablet.cc:1722] T e3b33c4d189247a89878b6a8e1fcbac0 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251028 03:00:04.523885 8544 tablet_bootstrap.cc:492] T c20410feb13543db9642c1e40bfb9300 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.524073 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc2faa9c4f04fe3b931ce8a2808d738" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.524163 8399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cdb02fb1b994c1bbfb18543b9776b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.524863 8391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf3884d16ec4706b75051f72d96a3b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.524904 8544 tablet_bootstrap.cc:654] T c20410feb13543db9642c1e40bfb9300 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.526821 8388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b6577bf30404cd6b27e0b9f17019137. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.527529 8394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af83fc5f22944800b924c2ee8c2a3903. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.528112 8545 tablet_bootstrap.cc:654] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.524176 8421 raft_consensus.cc:3060] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.528849 8390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece8bffc9f8347e6bf87808d3a61fb7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.530759 8395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04412f0a327243e895a555334d680c13. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.531026 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc2faa9c4f04fe3b931ce8a2808d738" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.531999 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fc2faa9c4f04fe3b931ce8a2808d738" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:04.532840 8083 leader_election.cc:343] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:04.533092 8083 leader_election.cc:343] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.536201 8545 tablet_bootstrap.cc:492] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.536300 8545 ts_tablet_manager.cc:1403] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.002s
I20251028 03:00:04.536799 8545 raft_consensus.cc:359] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.536885 8545 raft_consensus.cc:385] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.536921 8545 raft_consensus.cc:740] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.537014 8545 consensus_queue.cc:260] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.537346 7960 master_service.cc:954] ReplaceTablet: received request to replace tablet 23790dc59186409b9cc9f4d6cd2d2e4e from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:04.537462 8545 ts_tablet_manager.cc:1434] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.537640 8545 tablet_bootstrap.cc:492] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.538872 8545 tablet_bootstrap.cc:654] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.539409 8421 raft_consensus.cc:2468] T 2fc2faa9c4f04fe3b931ce8a2808d738 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:04.539848 8083 leader_election.cc:304] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.540063 10460 raft_consensus.cc:2804] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.540148 10460 raft_consensus.cc:697] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:04.540380 10460 consensus_queue.cc:237] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.541759 8382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2278777b0ffe446cb1090962c27819ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.542024 8383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e339967daa42a490678deea80c1c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20251028 03:00:04.543361 8551 tablet_replica.cc:333] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.543504 8551 raft_consensus.cc:2243] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.543601 8551 raft_consensus.cc:2272] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.544116 8551 ts_tablet_manager.cc:1916] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:04.544466 7963 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c5aa5e6d12eb4140aa803173cfae931e consensus state has no leader
I20251028 03:00:04.545301 7964 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_dde8663613354ea18dab6687dc0cbfeb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.545744 7964 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dde8663613354ea18dab6687dc0cbfeb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.546523 8551 ts_tablet_manager.cc:1929] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.546587 8551 log.cc:1199] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/d60fc654f6af44b290d46235082a4113
I20251028 03:00:04.546630 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2c98fdd4e565487c85c1d9128125e212" } modify_external_catalogs: true
I20251028 03:00:04.546736 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2c98fdd4e565487c85c1d9128125e212" } modify_external_catalogs: true
I20251028 03:00:04.546808 8551 ts_tablet_manager.cc:1950] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.547202 8551 tablet_replica.cc:333] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.548192 7967 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1e52c58d3e4042139159ba0e312a4089" } modify_external_catalogs: true
I20251028 03:00:04.548280 7967 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_1e52c58d3e4042139159ba0e312a4089" } modify_external_catalogs: true
I20251028 03:00:04.548264 8553 raft_consensus.cc:2749] T 26a60624333d43f1b701036b7e9dfdb0 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.550120 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 6dd563e13e624d4bb88950b46d480876 (table default.Table_0d32d1fdb9bb4294a8b7054819d111e5 [id=eaebbf79fab746dfaa81407d3509306c]) successfully deleted
I20251028 03:00:04.550323 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet d60fc654f6af44b290d46235082a4113 (table default.Table_a387569232e54415a0dd77e906ddcee4 [id=94895dadf0e34a19a454e8755c61c785]) successfully deleted
W20251028 03:00:04.550858 8356 leader_election.cc:343] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Not found: Tablet not found: d60fc654f6af44b290d46235082a4113
W20251028 03:00:04.552372 8356 consensus_peers.cc:597] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac782a9f0b564667a7b20f1e328f2572. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.556876 7963 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e5384e4fe2664224b92568f45ffc15c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.557339 7963 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e5384e4fe2664224b92568f45ffc15c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.559500 8551 raft_consensus.cc:2243] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.559796 8551 raft_consensus.cc:2272] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.560204 8551 ts_tablet_manager.cc:1916] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.562916 8551 ts_tablet_manager.cc:1929] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:04.562991 8551 log.cc:1199] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/ac782a9f0b564667a7b20f1e328f2572
I20251028 03:00:04.563251 8551 ts_tablet_manager.cc:1950] T ac782a9f0b564667a7b20f1e328f2572 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.563903 8551 tablet_replica.cc:333] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.564051 8551 raft_consensus.cc:2243] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.564054 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet ac782a9f0b564667a7b20f1e328f2572 (table default.Table_dddd9c465f0b4e4cad048c3b984cbab3 [id=11719def996f48e68ac7d4ee2dbf66c5]) successfully deleted
I20251028 03:00:04.564234 8551 pending_rounds.cc:70] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 03:00:04.564265 8551 pending_rounds.cc:77] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597175039197184 op_type: NO_OP noop_request { }
I20251028 03:00:04.564320 8551 raft_consensus.cc:2889] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:04.564371 8551 raft_consensus.cc:2272] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.566483 8550 ts_tablet_manager.cc:1929] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:04.567243 8546 tablet_bootstrap.cc:492] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.567370 8546 ts_tablet_manager.cc:1403] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.083s user 0.002s sys 0.000s
I20251028 03:00:04.567591 7961 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_3fb2a054809d4428abf6fd4cee9a79cf" } new_table_name: "default.Table_1d7c9db07cd3418087fdedd0d81d321d" modify_external_catalogs: true
I20251028 03:00:04.567905 8546 raft_consensus.cc:359] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.568017 8546 raft_consensus.cc:385] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.568053 8546 raft_consensus.cc:740] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.568181 8546 consensus_queue.cc:260] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.568502 8546 ts_tablet_manager.cc:1434] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.568873 8550 log.cc:1199] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/6f6f58ddaa6940248065a05b75996112
W20251028 03:00:04.569252 8354 leader_election.cc:343] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: d60fc654f6af44b290d46235082a4113
I20251028 03:00:04.569345 8354 leader_election.cc:304] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.569397 8550 ts_tablet_manager.cc:1950] T 6f6f58ddaa6940248065a05b75996112 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 03:00:04.569444 8354 raft_consensus.cc:2686] T d60fc654f6af44b290d46235082a4113 P d7e4cf694f71438298dd32a346abcf75: unable to run election callback: Service unavailable: Thread pool token was shut down
I20251028 03:00:04.569855 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 6f6f58ddaa6940248065a05b75996112 (table default.Table_6088be97ba4b4513b640257c2d69675a [id=b7ed75132db44133a09d9c0486be4706]) successfully deleted
I20251028 03:00:04.570387 8544 tablet_bootstrap.cc:492] T c20410feb13543db9642c1e40bfb9300 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.570482 8544 ts_tablet_manager.cc:1403] T c20410feb13543db9642c1e40bfb9300 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20251028 03:00:04.571040 8544 raft_consensus.cc:359] T c20410feb13543db9642c1e40bfb9300 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.571147 8544 raft_consensus.cc:385] T c20410feb13543db9642c1e40bfb9300 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.571174 8544 raft_consensus.cc:740] T c20410feb13543db9642c1e40bfb9300 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.571280 8544 consensus_queue.cc:260] T c20410feb13543db9642c1e40bfb9300 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.571584 8544 ts_tablet_manager.cc:1434] T c20410feb13543db9642c1e40bfb9300 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.571779 8544 tablet_bootstrap.cc:492] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.571907 8546 tablet_bootstrap.cc:492] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.573051 8546 tablet_bootstrap.cc:654] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.515280 7962 catalog_manager.cc:5649] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129). New cstate: current_term: 1 leader_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20251028 03:00:04.574570 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 58268ac590f3437a942dcafd927a5496 (table default.Table_0b7b98e5884a48de833ab1fbbad97f08 [id=fdb289a561dd4c6497173ef43e2e9ccd]). Expected version 1 got 0
I20251028 03:00:04.574769 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet 2fc2faa9c4f04fe3b931ce8a2808d738 (table default.Table_0d32d1fdb9bb4294a8b7054819d111e5 [id=eaebbf79fab746dfaa81407d3509306c]). Expected version 1 got 0
I20251028 03:00:04.574951 7962 catalog_manager.cc:5741] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529) does not have the latest schema for tablet a38ef10c145a4d00a26265a9dee5c744 (table default.Table_8d72f9b8a87847d0a40f66c2bcdba651 [id=ba8b6a629ebc489abe7043890455765a]). Expected version 1 got 0
I20251028 03:00:04.575423 10422 tablet.cc:1722] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:04.576027 8546 tablet_bootstrap.cc:492] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.576119 8546 ts_tablet_manager.cc:1403] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251028 03:00:04.576956 8544 tablet_bootstrap.cc:654] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.576900 8546 raft_consensus.cc:359] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.577004 8546 raft_consensus.cc:385] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.577037 8546 raft_consensus.cc:740] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.577194 8546 consensus_queue.cc:260] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.577499 8546 ts_tablet_manager.cc:1434] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20251028 03:00:04.579571 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet 6455fca4d6684904a255751ce63258f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6455fca4d6684904a255751ce63258f0 already in progress: creating tablet
I20251028 03:00:04.580264 8551 ts_tablet_manager.cc:1916] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.580952 7965 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_41dffdb5f3d848db840c0e7174dc3eac [id=a82bb1432b8546188bb85f1405f123ed] alter complete (version 1)
I20251028 03:00:04.581676 10423 tablet.cc:1722] T 6bd72bb34fe843079f8a4b828770e441 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20251028 03:00:04.581997 7953 catalog_manager.cc:4965] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): delete failed for tablet b2307c69e7ee47cbac31c54be957f5d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2307c69e7ee47cbac31c54be957f5d4 already in progress: creating tablet
I20251028 03:00:04.583920 10187 tablet.cc:1722] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:04.584404 8546 tablet_bootstrap.cc:492] T f62b981b9b1a44bb96bbc90f924c3b64 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.585526 8546 tablet_bootstrap.cc:654] T f62b981b9b1a44bb96bbc90f924c3b64 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.585529 8550 tablet_replica.cc:333] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.585680 8550 raft_consensus.cc:2243] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.589897 8551 ts_tablet_manager.cc:1929] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.589989 8551 log.cc:1199] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4b787f8f024745b297b2ebaa456cbcbf
I20251028 03:00:04.590245 8551 ts_tablet_manager.cc:1950] T 4b787f8f024745b297b2ebaa456cbcbf P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.590598 7969 master_service.cc:954] ReplaceTablet: received request to replace tablet 338cf9d02142436f97ccd44313cc8199 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:04.590910 8551 tablet_replica.cc:333] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.591085 8551 raft_consensus.cc:2243] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.591181 8551 raft_consensus.cc:2272] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.591856 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4b787f8f024745b297b2ebaa456cbcbf (table default.Table_35aa9cb3afdc4a149341fa51b840258f [id=a05d3a9fbe8c4278bcab247dcd663653]) successfully deleted
I20251028 03:00:04.591897 8545 tablet_bootstrap.cc:492] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.591975 8545 ts_tablet_manager.cc:1403] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.054s user 0.002s sys 0.000s
I20251028 03:00:04.592427 8545 raft_consensus.cc:359] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.592538 8545 raft_consensus.cc:385] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.592571 8545 raft_consensus.cc:740] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.592686 8545 consensus_queue.cc:260] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.592984 8545 ts_tablet_manager.cc:1434] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.593173 8545 tablet_bootstrap.cc:492] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.593938 8553 raft_consensus.cc:493] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.594177 8545 tablet_bootstrap.cc:654] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.594141 8553 raft_consensus.cc:515] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.594861 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f90e3c3524e4498aa6c4bfc5304d44b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.595244 8553 leader_election.cc:290] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.596038 9190 consensus_queue.cc:1048] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.598770 10460 consensus_queue.cc:1048] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.600744 8550 raft_consensus.cc:2272] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.601207 8550 ts_tablet_manager.cc:1916] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.601366 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4f90e3c3524e4498aa6c4bfc5304d44b" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.601519 8142 raft_consensus.cc:2393] T 4f90e3c3524e4498aa6c4bfc5304d44b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccbdf478f18948d19de96574a685bfda in current term 1: Already voted for candidate d7e4cf694f71438298dd32a346abcf75 in this term.
I20251028 03:00:04.602495 8218 leader_election.cc:304] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.602631 10465 raft_consensus.cc:3060] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.603869 10465 raft_consensus.cc:2749] T 4f90e3c3524e4498aa6c4bfc5304d44b P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.604174 10245 raft_consensus.cc:493] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.604264 10245 raft_consensus.cc:515] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.604601 10245 leader_election.cc:290] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.604773 7966 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_4b231be36bff4ad1b7a63b650b3a3ae1 [id=852bae9da8c24f93898f6e648067b1e0] alter complete (version 1)
I20251028 03:00:04.606174 8252 tablet_service.cc:1552] Processing DeleteTablet for tablet 3b0c593a3b96434db9ac42d260fc6997 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.606312 8258 tablet_service.cc:1552] Processing DeleteTablet for tablet 3435803c6167447a8b3afba6c0f3511b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.606348 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 8483cbc3099644d2b6f7ff52869e21ea with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.606782 8552 tablet_replica.cc:333] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
W20251028 03:00:04.606832 8083 consensus_peers.cc:597] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6dd563e13e624d4bb88950b46d480876. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.606930 8552 raft_consensus.cc:2243] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.607048 8552 raft_consensus.cc:2272] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.607162 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "39f0a9883b584083b542d078ea7c165b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:04.607411 7955 catalog_manager.cc:4952] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 3b0c593a3b96434db9ac42d260fc6997 because the tablet was not found. No further retry: Not found: Tablet not found: 3b0c593a3b96434db9ac42d260fc6997
W20251028 03:00:04.607766 8356 leader_election.cc:343] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.607856 8552 ts_tablet_manager.cc:1916] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.610131 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "39f0a9883b584083b542d078ea7c165b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.610260 8142 raft_consensus.cc:2468] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:04.612058 8552 ts_tablet_manager.cc:1929] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.612146 8552 log.cc:1199] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/3435803c6167447a8b3afba6c0f3511b
I20251028 03:00:04.612367 8552 ts_tablet_manager.cc:1950] T 3435803c6167447a8b3afba6c0f3511b P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.612984 8552 tablet_replica.cc:333] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.613130 8552 raft_consensus.cc:2243] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.613234 8552 raft_consensus.cc:2272] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.613639 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 23790dc59186409b9cc9f4d6cd2d2e4e
I20251028 03:00:04.614043 7960 catalog_manager.cc:6719] ReplaceTablet: tablet 23790dc59186409b9cc9f4d6cd2d2e4e deleted and replaced by tablet a2619de24ef643b5aa145f3f70160827
I20251028 03:00:04.614326 8552 ts_tablet_manager.cc:1916] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.614363 10465 consensus_queue.cc:1048] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.614836 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 3435803c6167447a8b3afba6c0f3511b (table default.Table_e40f83dab24e4e83b634a88cfe208817 [id=b987718414e246efb3353a3c8bd35f2f]) successfully deleted
I20251028 03:00:04.615353 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 23790dc59186409b9cc9f4d6cd2d2e4e with delete_type TABLET_DATA_DELETED (replaced by tablet a2619de24ef643b5aa145f3f70160827) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.616709 8382 tablet_service.cc:1552] Processing DeleteTablet for tablet 59cd8e6357fe4a6cae6be04c493a6880 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.617690 8382 tablet_service.cc:1552] Processing DeleteTablet for tablet f1d727c3dbef467da0aec436ffc393be with delete_type TABLET_DATA_DELETED (replaced by tablet a0943be517fd49e199fd060782a22c31) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.618587 8553 raft_consensus.cc:493] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.618672 8553 raft_consensus.cc:515] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.619200 8553 leader_election.cc:290] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 03:00:04.619593 8220 leader_election.cc:343] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 374936c130c841348e0d76cce070a295
I20251028 03:00:04.619855 8382 tablet_service.cc:1552] Processing DeleteTablet for tablet 4393587f022446828e344a9c394a92f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.620155 8382 tablet_service.cc:1552] Processing DeleteTablet for tablet 23790dc59186409b9cc9f4d6cd2d2e4e with delete_type TABLET_DATA_DELETED (replaced by tablet a2619de24ef643b5aa145f3f70160827) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.620321 8551 ts_tablet_manager.cc:1916] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.620438 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 23790dc59186409b9cc9f4d6cd2d2e4e with delete_type TABLET_DATA_DELETED (replaced by tablet a2619de24ef643b5aa145f3f70160827) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.621407 7966 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bc39558d01c04a3a93232e69380bf987" } new_table_name: "default.Table_b4925d6059544a69a92d4a211056abbf" modify_external_catalogs: true
I20251028 03:00:04.622067 8354 leader_election.cc:304] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.622227 10245 raft_consensus.cc:2804] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.622284 10245 raft_consensus.cc:493] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.622326 10245 raft_consensus.cc:3060] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.622828 7960 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_f4742a7359b54dd5b69eccfcab766d23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.623544 7960 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4742a7359b54dd5b69eccfcab766d23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.625036 10245 raft_consensus.cc:515] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.625481 10245 leader_election.cc:290] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:04.626080 8218 leader_election.cc:343] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 374936c130c841348e0d76cce070a295
I20251028 03:00:04.626048 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "39f0a9883b584083b542d078ea7c165b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:04.626169 8218 leader_election.cc:304] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.626317 8553 raft_consensus.cc:2749] T 374936c130c841348e0d76cce070a295 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.626603 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "39f0a9883b584083b542d078ea7c165b" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 03:00:04.626631 8356 leader_election.cc:343] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.626740 8142 raft_consensus.cc:3060] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.627388 8545 tablet_bootstrap.cc:492] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.627480 8545 ts_tablet_manager.cc:1403] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20251028 03:00:04.627810 8142 raft_consensus.cc:2468] T 39f0a9883b584083b542d078ea7c165b P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:04.628038 8545 raft_consensus.cc:359] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.628149 8545 raft_consensus.cc:385] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.628182 8545 raft_consensus.cc:740] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.628293 8545 consensus_queue.cc:260] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.628597 8545 ts_tablet_manager.cc:1434] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.628813 8545 tablet_bootstrap.cc:492] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.629909 8544 tablet_bootstrap.cc:492] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.629998 8544 ts_tablet_manager.cc:1403] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.058s user 0.001s sys 0.002s
I20251028 03:00:04.630034 8545 tablet_bootstrap.cc:654] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.630539 8544 raft_consensus.cc:359] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.630647 8544 raft_consensus.cc:385] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.630677 8544 raft_consensus.cc:740] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.630795 8544 consensus_queue.cc:260] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.631081 8544 ts_tablet_manager.cc:1434] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.631259 8544 tablet_bootstrap.cc:492] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.632264 8544 tablet_bootstrap.cc:654] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.633338 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet c21f794ec1284f5d99521954ed7ba9c9
I20251028 03:00:04.634722 10464 raft_consensus.cc:493] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.634814 10464 raft_consensus.cc:515] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.635150 10464 leader_election.cc:290] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.635517 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6818c3f2dfa48448bb2d9b38ed0dc19" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.635619 7968 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_8a0436d7986444c3bc7567eeb19e1672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.636056 7968 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a0436d7986444c3bc7567eeb19e1672 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.636857 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet c21f794ec1284f5d99521954ed7ba9c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.637389 8354 leader_election.cc:304] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.637562 9190 raft_consensus.cc:2804] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.637655 9190 raft_consensus.cc:697] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:04.637801 9190 consensus_queue.cc:237] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.638381 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet c21f794ec1284f5d99521954ed7ba9c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.638978 8546 tablet_bootstrap.cc:492] T f62b981b9b1a44bb96bbc90f924c3b64 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.639091 8546 ts_tablet_manager.cc:1403] T f62b981b9b1a44bb96bbc90f924c3b64 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.000s
I20251028 03:00:04.639642 8546 raft_consensus.cc:359] T f62b981b9b1a44bb96bbc90f924c3b64 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.639750 8546 raft_consensus.cc:385] T f62b981b9b1a44bb96bbc90f924c3b64 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.639780 8546 raft_consensus.cc:740] T f62b981b9b1a44bb96bbc90f924c3b64 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.639909 8546 consensus_queue.cc:260] T f62b981b9b1a44bb96bbc90f924c3b64 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.640208 8546 ts_tablet_manager.cc:1434] T f62b981b9b1a44bb96bbc90f924c3b64 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.640384 8546 tablet_bootstrap.cc:492] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.641387 8546 tablet_bootstrap.cc:654] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.642200 8550 ts_tablet_manager.cc:1929] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.642269 8550 log.cc:1199] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/2fec0adb79d04469b90c1ea2a5e9ba5b
I20251028 03:00:04.642489 8550 ts_tablet_manager.cc:1950] T 2fec0adb79d04469b90c1ea2a5e9ba5b P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.642652 8382 tablet_service.cc:1552] Processing DeleteTablet for tablet c21f794ec1284f5d99521954ed7ba9c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.642884 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 2fec0adb79d04469b90c1ea2a5e9ba5b (table default.Table_cc716ae9a4cd4b939b2e461342f03331 [id=7375e54ec4fc4747bf8e8db863ac69ff]) successfully deleted
I20251028 03:00:04.645704 8552 ts_tablet_manager.cc:1929] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.645903 8552 log.cc:1199] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/8483cbc3099644d2b6f7ff52869e21ea
I20251028 03:00:04.646199 8552 ts_tablet_manager.cc:1950] T 8483cbc3099644d2b6f7ff52869e21ea P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.646188 8142 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f6818c3f2dfa48448bb2d9b38ed0dc19" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.647225 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 8483cbc3099644d2b6f7ff52869e21ea (table default.Table_8d295e2ce9374929ab925a8d8329920d [id=de05ccfb39064421868853d6d9866bda]) successfully deleted
I20251028 03:00:04.647809 8270 raft_consensus.cc:3060] T f62b981b9b1a44bb96bbc90f924c3b64 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:04.648243 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 23790dc59186409b9cc9f4d6cd2d2e4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23790dc59186409b9cc9f4d6cd2d2e4e already in progress: creating tablet
I20251028 03:00:04.653091 8218 leader_election.cc:304] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.653374 10465 consensus_queue.cc:1048] T 4e4d980617994db4a188a98bb31eff92 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 03:00:04.653496 8083 consensus_peers.cc:597] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.653659 10465 raft_consensus.cc:2749] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.654628 10460 consensus_queue.cc:1048] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.659611 8552 tablet_replica.cc:333] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.659876 8552 raft_consensus.cc:2243] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.661361 8544 tablet_bootstrap.cc:492] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.661476 8544 ts_tablet_manager.cc:1403] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20251028 03:00:04.661693 8552 raft_consensus.cc:2272] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.662003 8544 raft_consensus.cc:359] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.662112 8544 raft_consensus.cc:385] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.662144 8544 raft_consensus.cc:740] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.662256 8544 consensus_queue.cc:260] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.662400 8552 ts_tablet_manager.cc:1916] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.662580 8544 ts_tablet_manager.cc:1434] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.662840 8544 tablet_bootstrap.cc:492] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.663276 8274 raft_consensus.cc:3060] T 58268ac590f3437a942dcafd927a5496 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.663892 8544 tablet_bootstrap.cc:654] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.661480 10460 raft_consensus.cc:493] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.664769 10469 consensus_queue.cc:1048] T 58268ac590f3437a942dcafd927a5496 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.664836 10460 raft_consensus.cc:515] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.665078 8552 ts_tablet_manager.cc:1929] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:04.665140 8552 log.cc:1199] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/c21f794ec1284f5d99521954ed7ba9c9
I20251028 03:00:04.665242 10460 leader_election.cc:290] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.665398 8552 ts_tablet_manager.cc:1950] T c21f794ec1284f5d99521954ed7ba9c9 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.665815 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59cd8e6357fe4a6cae6be04c493a6880" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:04.666110 8083 leader_election.cc:343] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.666131 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet c21f794ec1284f5d99521954ed7ba9c9 (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]) successfully deleted
I20251028 03:00:04.667407 8545 tablet_bootstrap.cc:492] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.667506 8545 ts_tablet_manager.cc:1403] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20251028 03:00:04.667423 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59cd8e6357fe4a6cae6be04c493a6880" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.667662 8418 raft_consensus.cc:2468] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:04.668025 8545 raft_consensus.cc:359] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.668123 8545 raft_consensus.cc:385] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.668160 8545 raft_consensus.cc:740] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.668306 8545 consensus_queue.cc:260] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.668665 8545 ts_tablet_manager.cc:1434] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.668860 8545 tablet_bootstrap.cc:492] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.668910 8083 leader_election.cc:304] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.669279 10469 raft_consensus.cc:2804] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.669339 10469 raft_consensus.cc:493] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.669373 10469 raft_consensus.cc:3060] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.669936 8545 tablet_bootstrap.cc:654] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.670385 10469 raft_consensus.cc:515] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.670730 10469 leader_election.cc:290] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.671155 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59cd8e6357fe4a6cae6be04c493a6880" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:04.671512 8083 leader_election.cc:343] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.671676 8551 ts_tablet_manager.cc:1929] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.671769 8551 log.cc:1199] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/3b0c593a3b96434db9ac42d260fc6997
I20251028 03:00:04.660882 8546 tablet_bootstrap.cc:492] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.672106 8546 ts_tablet_manager.cc:1403] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.000s
I20251028 03:00:04.672797 7967 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 583d6226a5f44d47a5ea4788c2b0d5ce
I20251028 03:00:04.673060 8546 raft_consensus.cc:359] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.673166 8546 raft_consensus.cc:385] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.673211 8546 raft_consensus.cc:740] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.673372 8551 ts_tablet_manager.cc:1950] T 3b0c593a3b96434db9ac42d260fc6997 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.673302 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_9929c1ca65774a4d9c86763313a1c1d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:04.673362 8546 consensus_queue.cc:260] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
W20251028 03:00:04.673755 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9929c1ca65774a4d9c86763313a1c1d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.673825 8546 ts_tablet_manager.cc:1434] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:04.674050 8546 tablet_bootstrap.cc:492] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.674543 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 583d6226a5f44d47a5ea4788c2b0d5ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.675343 8546 tablet_bootstrap.cc:654] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.677103 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 583d6226a5f44d47a5ea4788c2b0d5ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
W20251028 03:00:04.677196 7961 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5ce070da42d94fdba260478115c617a9 consensus state has no leader
I20251028 03:00:04.677670 8382 tablet_service.cc:1552] Processing DeleteTablet for tablet 583d6226a5f44d47a5ea4788c2b0d5ce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.677696 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_da496449ac3c4138bf76eb8f7ae5ad6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.678120 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da496449ac3c4138bf76eb8f7ae5ad6f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.678387 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 3b0c593a3b96434db9ac42d260fc6997 (table default.Table_6e2c063c779e4b639714776f91d8e559 [id=74ec1bf45d4a4f9ba045968dc2c38336]) successfully deleted
I20251028 03:00:04.679070 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ba3d1f9ba1214c1d8ef3d9f737e91d6e" } modify_external_catalogs: true
I20251028 03:00:04.679168 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_ba3d1f9ba1214c1d8ef3d9f737e91d6e" } modify_external_catalogs: true
I20251028 03:00:04.679611 9190 consensus_queue.cc:1048] T 4f90e3c3524e4498aa6c4bfc5304d44b P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.680855 8551 tablet_replica.cc:333] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.681054 8551 raft_consensus.cc:2243] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.681203 8551 raft_consensus.cc:2272] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.681272 8420 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59cd8e6357fe4a6cae6be04c493a6880" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.681384 8420 raft_consensus.cc:3060] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.681666 8551 ts_tablet_manager.cc:1916] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.682462 7969 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 338cf9d02142436f97ccd44313cc8199
I20251028 03:00:04.682476 8420 raft_consensus.cc:2468] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:04.682657 7968 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2e67a0ecccde42c8bff633eeba8af217" } modify_external_catalogs: true
I20251028 03:00:04.682744 7969 catalog_manager.cc:6719] ReplaceTablet: tablet 338cf9d02142436f97ccd44313cc8199 deleted and replaced by tablet f7478e55bb3a4a35aa6430d95a447bb6
I20251028 03:00:04.682794 7968 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_2e67a0ecccde42c8bff633eeba8af217" } modify_external_catalogs: true
I20251028 03:00:04.682905 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a27ed96bb5934fbd98f3015a38eff495" } new_table_name: "default.Table_b932e80866ed4e26be51d039f34a9fdb" modify_external_catalogs: true
W20251028 03:00:04.683652 7966 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e9b5228d4131446f8460917e961b8f15 consensus state has no leader
I20251028 03:00:04.683902 8545 tablet_bootstrap.cc:492] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.683995 8545 ts_tablet_manager.cc:1403] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20251028 03:00:04.684537 8545 raft_consensus.cc:359] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.684640 8545 raft_consensus.cc:385] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.684672 8545 raft_consensus.cc:740] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.684689 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 338cf9d02142436f97ccd44313cc8199 with delete_type TABLET_DATA_DELETED (replaced by tablet f7478e55bb3a4a35aa6430d95a447bb6) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.684791 8545 consensus_queue.cc:260] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.685117 8545 ts_tablet_manager.cc:1434] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.685207 9190 raft_consensus.cc:493] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.685298 8545 tablet_bootstrap.cc:492] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.685273 9190 raft_consensus.cc:515] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.685298 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 338cf9d02142436f97ccd44313cc8199 with delete_type TABLET_DATA_DELETED (replaced by tablet f7478e55bb3a4a35aa6430d95a447bb6) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.685585 9190 leader_election.cc:290] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.686331 8545 tablet_bootstrap.cc:654] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.686357 8083 leader_election.cc:304] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.686721 10460 raft_consensus.cc:2804] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.686810 10460 raft_consensus.cc:697] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:04.686846 7965 catalog_manager.cc:5741] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189) does not have the latest schema for tablet d195958488bf401c8668116a97bf0f58 (table default.Table_8e38a9fe73744973bb30019aa837de23 [id=d95696d7d42c4d999f99b584a7232245]). Expected version 1 got 0
I20251028 03:00:04.686964 10460 consensus_queue.cc:237] T 59cd8e6357fe4a6cae6be04c493a6880 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.688727 8382 tablet_service.cc:1552] Processing DeleteTablet for tablet 338cf9d02142436f97ccd44313cc8199 with delete_type TABLET_DATA_DELETED (replaced by tablet f7478e55bb3a4a35aa6430d95a447bb6) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.689410 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f1d727c3dbef467da0aec436ffc393be" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.689801 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet a38ef10c145a4d00a26265a9dee5c744 (table default.Table_8d72f9b8a87847d0a40f66c2bcdba651 [id=ba8b6a629ebc489abe7043890455765a]). Expected version 1 got 0
I20251028 03:00:04.690512 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet 26a60624333d43f1b701036b7e9dfdb0 (table default.Table_1bcb25d3e7d34d09b41809bee9767f40 [id=7f9bfe24f9e24488bb02c120633dfcb3]). Expected version 1 got 0
I20251028 03:00:04.690692 7964 catalog_manager.cc:5741] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525) does not have the latest schema for tablet f6818c3f2dfa48448bb2d9b38ed0dc19 (table default.Table_73967d33588f4f02ab5e0010826ec5f6 [id=24424b561e55419cbc0d32089a83e21a]). Expected version 1 got 0
W20251028 03:00:04.690753 8356 leader_election.cc:343] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:04.691649 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 583d6226a5f44d47a5ea4788c2b0d5ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 583d6226a5f44d47a5ea4788c2b0d5ce already in progress: creating tablet
W20251028 03:00:04.691672 8083 consensus_peers.cc:597] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee1d686a0e4c47f6b214224047a55747. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.691890 8552 tablet_replica.cc:333] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.692050 8552 raft_consensus.cc:2243] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.692184 8552 raft_consensus.cc:2272] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.692489 7960 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8985e28c7f9d4c84aed8eed96c94a66c" } modify_external_catalogs: true
I20251028 03:00:04.692632 7960 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_8985e28c7f9d4c84aed8eed96c94a66c" } modify_external_catalogs: true
I20251028 03:00:04.692770 8552 ts_tablet_manager.cc:1916] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.695218 8550 tablet_replica.cc:333] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.695438 8550 raft_consensus.cc:2243] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.695636 8550 pending_rounds.cc:70] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 03:00:04.695708 8550 pending_rounds.cc:77] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597175456112640 op_type: NO_OP noop_request { }
I20251028 03:00:04.695897 8546 tablet_bootstrap.cc:492] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.696028 8546 ts_tablet_manager.cc:1403] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.001s
I20251028 03:00:04.696285 8550 raft_consensus.cc:2889] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:04.696393 8550 raft_consensus.cc:2272] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.696982 8550 ts_tablet_manager.cc:1916] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.699398 10465 raft_consensus.cc:493] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.699492 10465 raft_consensus.cc:515] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.699847 10465 leader_election.cc:290] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:04.700470 8220 leader_election.cc:343] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 6dd563e13e624d4bb88950b46d480876
W20251028 03:00:04.701635 8083 consensus_peers.cc:597] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 -> Peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Couldn't send request to peer d7e4cf694f71438298dd32a346abcf75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee1d686a0e4c47f6b214224047a55747. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.701947 8550 ts_tablet_manager.cc:1929] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.702041 8550 log.cc:1199] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/6dd563e13e624d4bb88950b46d480876
I20251028 03:00:04.702323 8550 ts_tablet_manager.cc:1950] T 6dd563e13e624d4bb88950b46d480876 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.702665 8550 tablet_replica.cc:333] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.702819 8550 raft_consensus.cc:2243] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.702936 8550 raft_consensus.cc:2272] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.702812 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet 6dd563e13e624d4bb88950b46d480876 (table default.Table_0d32d1fdb9bb4294a8b7054819d111e5 [id=eaebbf79fab746dfaa81407d3509306c]) successfully deleted
I20251028 03:00:04.703323 8550 ts_tablet_manager.cc:1916] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.703729 8546 raft_consensus.cc:359] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.703867 8546 raft_consensus.cc:385] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.703905 8546 raft_consensus.cc:740] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.704046 8546 consensus_queue.cc:260] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.704381 8546 ts_tablet_manager.cc:1434] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.008s user 0.000s sys 0.000s
I20251028 03:00:04.705355 10469 consensus_queue.cc:1048] T 26a60624333d43f1b701036b7e9dfdb0 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.707516 8546 tablet_bootstrap.cc:492] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.708658 8546 tablet_bootstrap.cc:654] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.708945 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_84540ca800c346fcbf6e253858982069"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.709328 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_84540ca800c346fcbf6e253858982069 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.712373 8551 ts_tablet_manager.cc:1929] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.712458 8551 log.cc:1199] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/d5c3b02db24d4dc48260b42841844938
I20251028 03:00:04.712659 8551 ts_tablet_manager.cc:1950] T d5c3b02db24d4dc48260b42841844938 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.713697 8551 tablet_replica.cc:333] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.713872 8551 raft_consensus.cc:2243] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.713976 8551 raft_consensus.cc:2272] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.714828 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet d5c3b02db24d4dc48260b42841844938 (table default.Table_aa03ff4bf2694e7eabc52863451cba52 [id=3a8e945f16744123a8583ec0db730658]) successfully deleted
I20251028 03:00:04.715008 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_9793aeb6adf545ffb637885cbc748cea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.715008 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 6455fca4d6684904a255751ce63258f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6455fca4d6684904a255751ce63258f0 already in progress: creating tablet
W20251028 03:00:04.715282 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet b2307c69e7ee47cbac31c54be957f5d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2307c69e7ee47cbac31c54be957f5d4 already in progress: creating tablet
W20251028 03:00:04.715448 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9793aeb6adf545ffb637885cbc748cea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.715955 8545 tablet_bootstrap.cc:492] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.716048 8545 ts_tablet_manager.cc:1403] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.000s
I20251028 03:00:04.718230 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 4b3047edf5a44514a23b689ee80ff1b4
I20251028 03:00:04.718436 7961 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_468bc1558c6843cdbdeb3a4c12ddcfdc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:04.718744 8545 raft_consensus.cc:359] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.718873 8545 raft_consensus.cc:385] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.718907 8545 raft_consensus.cc:740] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
W20251028 03:00:04.718904 7961 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_468bc1558c6843cdbdeb3a4c12ddcfdc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.719041 8545 consensus_queue.cc:260] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.719503 8545 ts_tablet_manager.cc:1434] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 03:00:04.719699 8545 tablet_bootstrap.cc:492] T f5e7e0614af04cc2a22130906d062a79 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.720979 8545 tablet_bootstrap.cc:654] T f5e7e0614af04cc2a22130906d062a79 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.721519 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b3047edf5a44514a23b689ee80ff1b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.721663 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b3047edf5a44514a23b689ee80ff1b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.722502 8382 tablet_service.cc:1552] Processing DeleteTablet for tablet 4b3047edf5a44514a23b689ee80ff1b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.724650 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f1d727c3dbef467da0aec436ffc393be" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.724794 8143 raft_consensus.cc:2468] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:04.725291 10465 raft_consensus.cc:493] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.725374 10465 raft_consensus.cc:515] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.725724 10465 leader_election.cc:290] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:04.726339 8220 leader_election.cc:343] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 597d792ed13941a2a895b0dc305614f1
W20251028 03:00:04.728508 8218 leader_election.cc:343] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 6dd563e13e624d4bb88950b46d480876
I20251028 03:00:04.728619 8218 leader_election.cc:304] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.728842 10465 raft_consensus.cc:2749] T 6dd563e13e624d4bb88950b46d480876 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.732342 7968 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 7ee583e318174f34b162c2a104a0a3a9
I20251028 03:00:04.733285 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ee583e318174f34b162c2a104a0a3a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.734561 8382 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ee583e318174f34b162c2a104a0a3a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.734622 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 7ee583e318174f34b162c2a104a0a3a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.734992 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_bc45a4bd94f144cba69034f07727ae37" } new_table_name: "default.Table_d9d4384416244a6dbbe007ceddd396c8" modify_external_catalogs: true
I20251028 03:00:04.736733 8354 leader_election.cc:304] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.736979 9190 raft_consensus.cc:2804] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.737054 9190 raft_consensus.cc:493] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.737092 9190 raft_consensus.cc:3060] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.738145 9190 raft_consensus.cc:515] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.738529 9190 leader_election.cc:290] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
W20251028 03:00:04.738713 8218 leader_election.cc:343] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 597d792ed13941a2a895b0dc305614f1
I20251028 03:00:04.738837 8218 leader_election.cc:304] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.739073 10465 raft_consensus.cc:2749] T 597d792ed13941a2a895b0dc305614f1 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.742663 9190 raft_consensus.cc:493] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.742762 9190 raft_consensus.cc:515] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.743139 9190 leader_election.cc:290] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.743249 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f1d727c3dbef467da0aec436ffc393be" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:04.743752 9190 raft_consensus.cc:493] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251028 03:00:04.743798 8356 leader_election.cc:343] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.743846 7963 master_service.cc:954] ReplaceTablet: received request to replace tablet 6bd72bb34fe843079f8a4b828770e441 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:04.743865 9190 raft_consensus.cc:515] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
W20251028 03:00:04.744208 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f6864bcffa17438eb39f90fce8acb18e consensus state has no leader
I20251028 03:00:04.744320 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d195958488bf401c8668116a97bf0f58" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.744215 9190 leader_election.cc:290] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:04.744768 8356 leader_election.cc:343] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.744889 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4393587f022446828e344a9c394a92f9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:04.745209 8356 leader_election.cc:343] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.747357 7969 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_a56dbd80138241e084ed7c36f85873c7" } new_table_name: "default.Table_ce5b7a1919174cb4ada30210c4459299" modify_external_catalogs: true
I20251028 03:00:04.749756 8544 tablet_bootstrap.cc:492] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.749867 8544 ts_tablet_manager.cc:1403] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.087s user 0.002s sys 0.000s
I20251028 03:00:04.750409 8544 raft_consensus.cc:359] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.750548 8544 raft_consensus.cc:385] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.750587 8544 raft_consensus.cc:740] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.750722 8544 consensus_queue.cc:260] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.751040 8544 ts_tablet_manager.cc:1434] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.751468 8551 ts_tablet_manager.cc:1916] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.751693 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f1d727c3dbef467da0aec436ffc393be" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:04.751804 8143 raft_consensus.cc:3060] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.752985 8143 raft_consensus.cc:2468] T f1d727c3dbef467da0aec436ffc393be P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:04.754237 8544 tablet_bootstrap.cc:492] T f5e7e0614af04cc2a22130906d062a79 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.754449 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d195958488bf401c8668116a97bf0f58" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.754873 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4393587f022446828e344a9c394a92f9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.754968 8143 raft_consensus.cc:2468] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:04.755225 8546 tablet_bootstrap.cc:492] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.755362 8546 ts_tablet_manager.cc:1403] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.000s
I20251028 03:00:04.755403 8544 tablet_bootstrap.cc:654] T f5e7e0614af04cc2a22130906d062a79 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.755692 8550 ts_tablet_manager.cc:1929] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.755955 8546 raft_consensus.cc:359] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.755985 8354 leader_election.cc:304] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.756143 8546 raft_consensus.cc:385] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.756222 8546 raft_consensus.cc:740] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.756582 9190 raft_consensus.cc:2804] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.756791 8546 consensus_queue.cc:260] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.756878 7965 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: default.Table_745438bc0c014b13ab3381877a959e1e [id=45e7b7c657e54d43a490443857404c3c] alter complete (version 1)
I20251028 03:00:04.757258 8546 ts_tablet_manager.cc:1434] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:04.757450 8546 tablet_bootstrap.cc:492] T d0dad91f926648c781d1582733d6ae95 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.757505 7965 master_service.cc:954] ReplaceTablet: received request to replace tablet 900973d832c24b62a4554ae9a219cfd8 from {username='slave'} at 127.0.0.1:47364
I20251028 03:00:04.758234 8552 ts_tablet_manager.cc:1929] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:04.758299 8552 log.cc:1199] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/338cf9d02142436f97ccd44313cc8199
I20251028 03:00:04.758564 8552 ts_tablet_manager.cc:1950] T 338cf9d02142436f97ccd44313cc8199 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.760331 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 338cf9d02142436f97ccd44313cc8199 (table default.Table_77a37bbdfa4f41458e6b50fb213850bc [id=5f61b3c12ed647968fc2932aa24a13f3]) successfully deleted
W20251028 03:00:04.760501 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 4b3047edf5a44514a23b689ee80ff1b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b3047edf5a44514a23b689ee80ff1b4 already in progress: creating tablet
W20251028 03:00:04.760635 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 7ee583e318174f34b162c2a104a0a3a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ee583e318174f34b162c2a104a0a3a9 already in progress: creating tablet
I20251028 03:00:04.760823 8546 tablet_bootstrap.cc:654] T d0dad91f926648c781d1582733d6ae95 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.760852 8382 tablet_service.cc:1552] Processing DeleteTablet for tablet 0ee59aeedd2c4c9ba94afe6578af649a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:58 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.760934 8383 tablet_service.cc:1552] Processing DeleteTablet for tablet 39f0a9883b584083b542d078ea7c165b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.760931 9190 raft_consensus.cc:697] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:04.761797 8545 tablet_bootstrap.cc:492] T f5e7e0614af04cc2a22130906d062a79 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.761919 8545 ts_tablet_manager.cc:1403] T f5e7e0614af04cc2a22130906d062a79 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.000s
I20251028 03:00:04.761272 9190 consensus_queue.cc:237] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.763809 8545 raft_consensus.cc:359] T f5e7e0614af04cc2a22130906d062a79 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.763983 8545 raft_consensus.cc:385] T f5e7e0614af04cc2a22130906d062a79 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.764070 8545 raft_consensus.cc:740] T f5e7e0614af04cc2a22130906d062a79 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.764276 8545 consensus_queue.cc:260] T f5e7e0614af04cc2a22130906d062a79 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
W20251028 03:00:04.764830 8354 leader_election.cc:343] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.764907 8354 leader_election.cc:304] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.765203 8545 ts_tablet_manager.cc:1434] T f5e7e0614af04cc2a22130906d062a79 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251028 03:00:04.765362 10245 raft_consensus.cc:2749] T d195958488bf401c8668116a97bf0f58 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.765398 8545 tablet_bootstrap.cc:492] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.765823 8354 leader_election.cc:304] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.766048 10242 raft_consensus.cc:2804] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.766093 10242 raft_consensus.cc:493] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.766126 10242 raft_consensus.cc:3060] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.766440 8545 tablet_bootstrap.cc:654] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.767174 10242 raft_consensus.cc:515] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.767554 10242 leader_election.cc:290] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.768229 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4393587f022446828e344a9c394a92f9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:04.769834 8356 leader_election.cc:343] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.774611 8553 consensus_queue.cc:1048] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.775709 8545 tablet_bootstrap.cc:492] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.775832 8545 ts_tablet_manager.cc:1403] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20251028 03:00:04.776320 8545 raft_consensus.cc:359] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.776425 8545 raft_consensus.cc:385] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.776456 8545 raft_consensus.cc:740] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.776563 8545 consensus_queue.cc:260] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.776855 8545 ts_tablet_manager.cc:1434] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:04.777036 8545 tablet_bootstrap.cc:492] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.778048 8545 tablet_bootstrap.cc:654] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.778043 7960 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6ea1ef71a0974a5da443a61dc419c811
I20251028 03:00:04.782938 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 6ea1ef71a0974a5da443a61dc419c811 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.783603 8383 tablet_service.cc:1552] Processing DeleteTablet for tablet 6ea1ef71a0974a5da443a61dc419c811 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.784476 8546 tablet_bootstrap.cc:492] T d0dad91f926648c781d1582733d6ae95 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.784626 8546 ts_tablet_manager.cc:1403] T d0dad91f926648c781d1582733d6ae95 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20251028 03:00:04.785285 8546 raft_consensus.cc:359] T d0dad91f926648c781d1582733d6ae95 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.785434 8546 raft_consensus.cc:385] T d0dad91f926648c781d1582733d6ae95 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.785493 8546 raft_consensus.cc:740] T d0dad91f926648c781d1582733d6ae95 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.785660 8546 consensus_queue.cc:260] T d0dad91f926648c781d1582733d6ae95 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.786068 8546 ts_tablet_manager.cc:1434] T d0dad91f926648c781d1582733d6ae95 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.788595 8551 ts_tablet_manager.cc:1929] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.788795 8551 log.cc:1199] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/59cd8e6357fe4a6cae6be04c493a6880
I20251028 03:00:04.789130 8551 ts_tablet_manager.cc:1950] T 59cd8e6357fe4a6cae6be04c493a6880 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.789944 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 59cd8e6357fe4a6cae6be04c493a6880 (table default.Table_d5216511b6cb4223afa3920eec378c6b [id=9a604bc56e0640f38093ad6d9d04930c]) successfully deleted
I20251028 03:00:04.790987 10422 tablet.cc:1722] T f422030a6a4042009efc7fcfb21a6bca P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:04.792913 8546 tablet_bootstrap.cc:492] T 8e202e53053248a7a9cee6a217999489 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.794181 8546 tablet_bootstrap.cc:654] T 8e202e53053248a7a9cee6a217999489 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:04.795244 7955 catalog_manager.cc:4965] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): delete failed for tablet 6ea1ef71a0974a5da443a61dc419c811 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ea1ef71a0974a5da443a61dc419c811 already in progress: creating tablet
I20251028 03:00:04.795346 10187 tablet.cc:1722] T f422030a6a4042009efc7fcfb21a6bca P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:04.798754 8550 log.cc:1199] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/e9ad4ae17665483584ce2d638d1639fb
I20251028 03:00:04.762426 10469 raft_consensus.cc:493] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.799154 8550 ts_tablet_manager.cc:1950] T e9ad4ae17665483584ce2d638d1639fb P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.799114 10469 raft_consensus.cc:515] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.799546 10469 leader_election.cc:290] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.764173 8121 tablet_service.cc:1552] Processing DeleteTablet for tablet 39f0a9883b584083b542d078ea7c165b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.764281 8118 tablet_service.cc:1552] Processing DeleteTablet for tablet 28b81f0000c4407096feff773c29579d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.764345 8127 tablet_service.cc:1552] Processing DeleteTablet for tablet ee2fd98744ff44028f36bb55e6eba51e with delete_type TABLET_DATA_DELETED (replaced by tablet adf43d587639446e8bb3959b490c931b) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.764427 8120 tablet_service.cc:1552] Processing DeleteTablet for tablet 6dd563e13e624d4bb88950b46d480876 with delete_type TABLET_DATA_DELETED (replaced by tablet 2fc2faa9c4f04fe3b931ce8a2808d738) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.764498 8125 tablet_service.cc:1552] Processing DeleteTablet for tablet 59cd8e6357fe4a6cae6be04c493a6880 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:59 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.764518 8111 tablet_service.cc:1552] Processing DeleteTablet for tablet 4393587f022446828e344a9c394a92f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:00 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.764737 8124 tablet_service.cc:1552] Processing DeleteTablet for tablet 5d5c68c16c3e4d37be7f9a505ab44549 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 02:59:57 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.777558 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4393587f022446828e344a9c394a92f9" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:04.799909 7953 catalog_manager.cc:4985] TS fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): tablet e9ad4ae17665483584ce2d638d1639fb (table default.Table_929b382a93794515a84b4f6356fb86b2 [id=5f35ae4ce3914890a3ce64198fbdb39e]) successfully deleted
I20251028 03:00:04.799911 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5d5c68c16c3e4d37be7f9a505ab44549" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.799948 8146 raft_consensus.cc:3060] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
W20251028 03:00:04.800264 8083 leader_election.cc:343] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.801084 8146 raft_consensus.cc:2468] T 4393587f022446828e344a9c394a92f9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:04.791759 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 6ea1ef71a0974a5da443a61dc419c811 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-28 03:00:04 UTC) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.802913 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5d5c68c16c3e4d37be7f9a505ab44549" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.803056 8419 raft_consensus.cc:2468] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:04.803683 8083 leader_election.cc:304] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.803958 10469 raft_consensus.cc:2804] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.804044 10469 raft_consensus.cc:493] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.804090 10469 raft_consensus.cc:3060] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.805132 10469 raft_consensus.cc:515] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.805519 10469 leader_election.cc:290] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.805963 10422 tablet.cc:1722] T f62b981b9b1a44bb96bbc90f924c3b64 P d7e4cf694f71438298dd32a346abcf75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 03:00:04.806016 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5d5c68c16c3e4d37be7f9a505ab44549" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:04.806586 8083 leader_election.cc:343] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.807451 10187 tablet.cc:1722] T f62b981b9b1a44bb96bbc90f924c3b64 P ccbdf478f18948d19de96574a685bfda: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 03:00:04.807873 8419 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5d5c68c16c3e4d37be7f9a505ab44549" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.807976 8419 raft_consensus.cc:3060] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.808557 8354 leader_election.cc:304] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.808763 9190 raft_consensus.cc:2804] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.808847 9190 raft_consensus.cc:697] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:04.808933 8419 raft_consensus.cc:2468] T 5d5c68c16c3e4d37be7f9a505ab44549 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:04.808995 9190 consensus_queue.cc:237] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.809484 7966 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_124af076e3de4c09a8f32be24fa78260"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.809994 7966 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_124af076e3de4c09a8f32be24fa78260 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.810029 8083 leader_election.cc:304] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.810256 10469 raft_consensus.cc:2804] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.810338 10469 raft_consensus.cc:697] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:04.810469 10469 consensus_queue.cc:237] T 5d5c68c16c3e4d37be7f9a505ab44549 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.814396 8550 tablet_replica.cc:333] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.815423 8550 raft_consensus.cc:2243] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.816596 10423 tablet.cc:1722] T f62b981b9b1a44bb96bbc90f924c3b64 P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20251028 03:00:04.822475 8546 tablet_bootstrap.cc:492] T 8e202e53053248a7a9cee6a217999489 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.822549 8550 raft_consensus.cc:2272] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.822731 8546 ts_tablet_manager.cc:1403] T 8e202e53053248a7a9cee6a217999489 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.001s
I20251028 03:00:04.822957 8550 ts_tablet_manager.cc:1916] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.825657 8546 raft_consensus.cc:359] T 8e202e53053248a7a9cee6a217999489 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.825910 8546 raft_consensus.cc:385] T 8e202e53053248a7a9cee6a217999489 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.826010 8546 raft_consensus.cc:740] T 8e202e53053248a7a9cee6a217999489 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.826188 8546 consensus_queue.cc:260] T 8e202e53053248a7a9cee6a217999489 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.826629 8546 ts_tablet_manager.cc:1434] T 8e202e53053248a7a9cee6a217999489 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20251028 03:00:04.826893 8546 tablet_bootstrap.cc:492] T f1d727c3dbef467da0aec436ffc393be P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 03:00:04.826936 8356 consensus_peers.cc:597] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 338cf9d02142436f97ccd44313cc8199. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.823220 8544 tablet_bootstrap.cc:492] T f5e7e0614af04cc2a22130906d062a79 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.827422 8544 ts_tablet_manager.cc:1403] T f5e7e0614af04cc2a22130906d062a79 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.073s user 0.000s sys 0.002s
I20251028 03:00:04.827916 8544 raft_consensus.cc:359] T f5e7e0614af04cc2a22130906d062a79 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.828032 8544 raft_consensus.cc:385] T f5e7e0614af04cc2a22130906d062a79 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.828074 8544 raft_consensus.cc:740] T f5e7e0614af04cc2a22130906d062a79 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.828186 8544 consensus_queue.cc:260] T f5e7e0614af04cc2a22130906d062a79 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.828491 8546 tablet_bootstrap.cc:654] T f1d727c3dbef467da0aec436ffc393be P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.828490 8544 ts_tablet_manager.cc:1434] T f5e7e0614af04cc2a22130906d062a79 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:04.828655 8544 tablet_bootstrap.cc:492] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.829840 8544 tablet_bootstrap.cc:654] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.833261 8550 ts_tablet_manager.cc:1929] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.834245 7967 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_af65cc04ea5f422b85e4413a8baa43b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.834705 7967 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af65cc04ea5f422b85e4413a8baa43b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.835762 9958 consensus_queue.cc:1048] T a578008e735a4cc1a2f50dba09c16117 P ccbdf478f18948d19de96574a685bfda [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.836412 10242 consensus_queue.cc:1048] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.842756 8551 tablet_replica.cc:333] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.843020 8551 raft_consensus.cc:2243] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.843251 8551 pending_rounds.cc:70] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 03:00:04.843389 8551 pending_rounds.cc:77] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597177909620736 op_type: NO_OP noop_request { }
I20251028 03:00:04.843537 8551 raft_consensus.cc:2889] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:04.843670 8551 raft_consensus.cc:2272] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.844904 8551 ts_tablet_manager.cc:1916] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.845324 7961 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_829c6eb42be14ebbb7b97fec16a89ba6" } modify_external_catalogs: true
I20251028 03:00:04.845458 7961 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_829c6eb42be14ebbb7b97fec16a89ba6" } modify_external_catalogs: true
I20251028 03:00:04.847731 8546 tablet_bootstrap.cc:492] T f1d727c3dbef467da0aec436ffc393be P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.847916 8546 ts_tablet_manager.cc:1403] T f1d727c3dbef467da0aec436ffc393be P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.001s
I20251028 03:00:04.848453 8546 raft_consensus.cc:359] T f1d727c3dbef467da0aec436ffc393be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.848644 8546 raft_consensus.cc:385] T f1d727c3dbef467da0aec436ffc393be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.848735 8546 raft_consensus.cc:740] T f1d727c3dbef467da0aec436ffc393be P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.848935 8546 consensus_queue.cc:260] T f1d727c3dbef467da0aec436ffc393be P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.849383 8546 ts_tablet_manager.cc:1434] T f1d727c3dbef467da0aec436ffc393be P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.849623 8546 tablet_bootstrap.cc:492] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
W20251028 03:00:04.849782 7968 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2851b9c52e144c85a5f6673a8723c289 consensus state has no leader
I20251028 03:00:04.850800 8546 tablet_bootstrap.cc:654] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.858934 8545 tablet_bootstrap.cc:492] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.859047 8545 ts_tablet_manager.cc:1403] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.082s user 0.002s sys 0.000s
I20251028 03:00:04.859567 8545 raft_consensus.cc:359] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.859676 8545 raft_consensus.cc:385] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.859709 8545 raft_consensus.cc:740] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.859838 8545 consensus_queue.cc:260] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.860144 8545 ts_tablet_manager.cc:1434] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:04.860338 8545 tablet_bootstrap.cc:492] T 07509e5c36d9440fa9fe29c251931c63 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.861433 8545 tablet_bootstrap.cc:654] T 07509e5c36d9440fa9fe29c251931c63 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
W20251028 03:00:04.864392 8356 consensus_peers.cc:597] T 3a5415a4246c47ca896751f2fd68af71 P d7e4cf694f71438298dd32a346abcf75 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20251028 03:00:04.865957 8546 tablet_bootstrap.cc:492] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.866104 7963 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 6bd72bb34fe843079f8a4b828770e441
I20251028 03:00:04.866149 8546 ts_tablet_manager.cc:1403] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20251028 03:00:04.866428 7963 catalog_manager.cc:6719] ReplaceTablet: tablet 6bd72bb34fe843079f8a4b828770e441 deleted and replaced by tablet 2d28ec23ab5d4c50ada253168a5fa76e
I20251028 03:00:04.866571 8383 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bd72bb34fe843079f8a4b828770e441 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d28ec23ab5d4c50ada253168a5fa76e) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.866648 8546 raft_consensus.cc:359] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.866752 8546 raft_consensus.cc:385] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.866779 8546 raft_consensus.cc:740] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.866917 8546 consensus_queue.cc:260] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.867252 8546 ts_tablet_manager.cc:1434] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.867450 8546 tablet_bootstrap.cc:492] T 3a5415a4246c47ca896751f2fd68af71 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.867954 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bd72bb34fe843079f8a4b828770e441 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d28ec23ab5d4c50ada253168a5fa76e) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.868952 8546 tablet_bootstrap.cc:654] T 3a5415a4246c47ca896751f2fd68af71 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.869073 8552 tablet_replica.cc:333] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.869210 8552 raft_consensus.cc:2243] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.869321 8552 raft_consensus.cc:2272] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.870056 8552 ts_tablet_manager.cc:1916] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.871631 8551 ts_tablet_manager.cc:1929] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.872709 8552 ts_tablet_manager.cc:1929] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 03:00:04.872849 8552 log.cc:1199] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/6bd72bb34fe843079f8a4b828770e441
I20251028 03:00:04.873181 8552 ts_tablet_manager.cc:1950] T 6bd72bb34fe843079f8a4b828770e441 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.873481 8551 log.cc:1199] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/f1d727c3dbef467da0aec436ffc393be
I20251028 03:00:04.874040 7955 catalog_manager.cc:4985] TS ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): tablet 6bd72bb34fe843079f8a4b828770e441 (table default.Table_a058dd240b5e45d68a76c604a5fb0ef6 [id=262048a24b0f47b0b3c0dee427389e4c]) successfully deleted
I20251028 03:00:04.874507 8551 ts_tablet_manager.cc:1950] T f1d727c3dbef467da0aec436ffc393be P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.875252 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet f1d727c3dbef467da0aec436ffc393be (table default.Table_fcb100729f5b48b990a9c4d2d9b9fcf2 [id=0381bc31864343c19a461fe49735d58a]) successfully deleted
I20251028 03:00:04.876645 8551 tablet_replica.cc:333] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.877030 8551 raft_consensus.cc:2243] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.877204 7963 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_abcc95f588d04951b016ffd9ea9800c5" } modify_external_catalogs: true
I20251028 03:00:04.877305 8551 pending_rounds.cc:70] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 03:00:04.877318 7963 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_abcc95f588d04951b016ffd9ea9800c5" } modify_external_catalogs: true
I20251028 03:00:04.877338 8551 pending_rounds.cc:77] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597178099240960 op_type: NO_OP noop_request { }
I20251028 03:00:04.877393 8551 raft_consensus.cc:2889] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:04.877460 8551 raft_consensus.cc:2272] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.878726 8546 tablet_bootstrap.cc:492] T 3a5415a4246c47ca896751f2fd68af71 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.879153 8546 ts_tablet_manager.cc:1403] T 3a5415a4246c47ca896751f2fd68af71 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.003s
I20251028 03:00:04.879894 8546 raft_consensus.cc:359] T 3a5415a4246c47ca896751f2fd68af71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.880090 8546 raft_consensus.cc:385] T 3a5415a4246c47ca896751f2fd68af71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.880158 8546 raft_consensus.cc:740] T 3a5415a4246c47ca896751f2fd68af71 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.880345 8546 consensus_queue.cc:260] T 3a5415a4246c47ca896751f2fd68af71 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.880692 8546 ts_tablet_manager.cc:1434] T 3a5415a4246c47ca896751f2fd68af71 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.880685 8551 ts_tablet_manager.cc:1916] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.880955 8546 tablet_bootstrap.cc:492] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.882076 8546 tablet_bootstrap.cc:654] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.883175 8551 ts_tablet_manager.cc:1929] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.883498 9595 raft_consensus.cc:493] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.883579 9595 raft_consensus.cc:515] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.883931 9595 leader_election.cc:290] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:04.884636 8220 leader_election.cc:343] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 2fec0adb79d04469b90c1ea2a5e9ba5b
W20251028 03:00:04.889539 7969 catalog_manager.cc:5909] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a5fa3e89d7994f35bfbf55378101a0d9 consensus state has no leader
I20251028 03:00:04.889863 7969 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_5f67c669af6046cb8e9ab5024d0b834e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251028 03:00:04.883773 8545 tablet_bootstrap.cc:492] T 07509e5c36d9440fa9fe29c251931c63 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.890115 8545 ts_tablet_manager.cc:1403] T 07509e5c36d9440fa9fe29c251931c63 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
W20251028 03:00:04.890313 7969 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f67c669af6046cb8e9ab5024d0b834e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.890726 8545 raft_consensus.cc:359] T 07509e5c36d9440fa9fe29c251931c63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.890830 8545 raft_consensus.cc:385] T 07509e5c36d9440fa9fe29c251931c63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.890862 8545 raft_consensus.cc:740] T 07509e5c36d9440fa9fe29c251931c63 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.890995 8545 consensus_queue.cc:260] T 07509e5c36d9440fa9fe29c251931c63 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.891350 8545 ts_tablet_manager.cc:1434] T 07509e5c36d9440fa9fe29c251931c63 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.891346 7968 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47364:
table { table_name: "default.Table_f2dee7552f5543b0ac6899b7a28431a2" } new_table_name: "default.Table_8c871a56a026425fa6d1d0703ae62b88" modify_external_catalogs: true
I20251028 03:00:04.891566 8545 tablet_bootstrap.cc:492] T f03b90d615ab435f9a244504d8d01660 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
W20251028 03:00:04.891696 8218 leader_election.cc:343] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: 2fec0adb79d04469b90c1ea2a5e9ba5b
I20251028 03:00:04.891801 8218 leader_election.cc:304] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.892036 9595 raft_consensus.cc:2749] T 2fec0adb79d04469b90c1ea2a5e9ba5b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.892614 8545 tablet_bootstrap.cc:654] T f03b90d615ab435f9a244504d8d01660 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.897532 10546 raft_consensus.cc:493] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.897632 10546 raft_consensus.cc:515] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.898339 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "698cca4847e94a79933c644f6b1c1c8d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.898478 10546 leader_election.cc:290] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:04.898698 8356 leader_election.cc:343] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.901535 8551 log.cc:1199] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/4393587f022446828e344a9c394a92f9
I20251028 03:00:04.901872 8551 ts_tablet_manager.cc:1950] T 4393587f022446828e344a9c394a92f9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.902619 8551 tablet_replica.cc:333] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.902650 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 4393587f022446828e344a9c394a92f9 (table default.Table_0ce46dd2a59848399a1d4fdeac7deef9 [id=e574f40b6ddb4eccae167d16875b0e7d]) successfully deleted
I20251028 03:00:04.902765 8551 raft_consensus.cc:2243] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.902871 8551 raft_consensus.cc:2272] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:04.903157 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 23790dc59186409b9cc9f4d6cd2d2e4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23790dc59186409b9cc9f4d6cd2d2e4e already in progress: creating tablet
I20251028 03:00:04.903249 8551 ts_tablet_manager.cc:1916] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.905380 8545 tablet_bootstrap.cc:492] T f03b90d615ab435f9a244504d8d01660 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.905450 8546 tablet_bootstrap.cc:492] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.905555 8546 ts_tablet_manager.cc:1403] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20251028 03:00:04.905571 8545 ts_tablet_manager.cc:1403] T f03b90d615ab435f9a244504d8d01660 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20251028 03:00:04.906128 8546 raft_consensus.cc:359] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.906260 8546 raft_consensus.cc:385] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.906168 8545 raft_consensus.cc:359] T f03b90d615ab435f9a244504d8d01660 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.906297 8546 raft_consensus.cc:740] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.906440 8546 consensus_queue.cc:260] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.906641 8545 raft_consensus.cc:385] T f03b90d615ab435f9a244504d8d01660 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.906728 8545 raft_consensus.cc:740] T f03b90d615ab435f9a244504d8d01660 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.906939 8545 consensus_queue.cc:260] T f03b90d615ab435f9a244504d8d01660 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.907542 8545 ts_tablet_manager.cc:1434] T f03b90d615ab435f9a244504d8d01660 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:04.907930 8545 tablet_bootstrap.cc:492] T 746eae30ca284758a6e3ed8ad80c2cdc P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.910013 8546 ts_tablet_manager.cc:1434] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251028 03:00:04.910279 8546 tablet_bootstrap.cc:492] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.913653 8545 tablet_bootstrap.cc:654] T 746eae30ca284758a6e3ed8ad80c2cdc P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.915182 8546 tablet_bootstrap.cc:654] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.835492 10423 tablet.cc:1722] T f422030a6a4042009efc7fcfb21a6bca P fbd2afb4894c4e7a8520cc09f8593aa0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251028 03:00:04.876824 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 6bd72bb34fe843079f8a4b828770e441 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d28ec23ab5d4c50ada253168a5fa76e) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.916993 9595 raft_consensus.cc:493] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.917084 9595 raft_consensus.cc:515] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.917475 9595 leader_election.cc:290] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.918020 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "463f1c3ac2d74791830431191601282c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.918146 8421 raft_consensus.cc:2468] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:04.918448 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "463f1c3ac2d74791830431191601282c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.918514 8220 leader_election.cc:304] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:04.918560 8146 raft_consensus.cc:2468] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 0.
I20251028 03:00:04.918669 9595 raft_consensus.cc:2804] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.918708 9595 raft_consensus.cc:493] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.918742 9595 raft_consensus.cc:3060] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.898795 8544 tablet_bootstrap.cc:492] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.919543 8544 ts_tablet_manager.cc:1403] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.091s user 0.001s sys 0.001s
I20251028 03:00:04.920068 8544 raft_consensus.cc:359] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.920176 8544 raft_consensus.cc:385] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.920210 8544 raft_consensus.cc:740] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.920352 8544 consensus_queue.cc:260] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.920646 8544 ts_tablet_manager.cc:1434] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.899170 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "698cca4847e94a79933c644f6b1c1c8d" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.920626 9595 raft_consensus.cc:515] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.920976 9595 leader_election.cc:290] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.921563 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "463f1c3ac2d74791830431191601282c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
W20251028 03:00:04.921595 8354 leader_election.cc:343] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.921666 8143 raft_consensus.cc:3060] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.921669 8354 leader_election.cc:304] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.921861 10242 raft_consensus.cc:2749] T 698cca4847e94a79933c644f6b1c1c8d P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.922386 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "463f1c3ac2d74791830431191601282c" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.922501 8421 raft_consensus.cc:3060] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.922767 8143 raft_consensus.cc:2468] T 463f1c3ac2d74791830431191601282c P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:04.923049 8218 leader_election.cc:304] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:04.923240 9595 raft_consensus.cc:2804] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.923347 9595 raft_consensus.cc:697] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Becoming Leader. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Running, Role: LEADER
I20251028 03:00:04.923468 9595 consensus_queue.cc:237] T 463f1c3ac2d74791830431191601282c P ccbdf478f18948d19de96574a685bfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.923554 8421 raft_consensus.cc:2468] T 463f1c3ac2d74791830431191601282c P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccbdf478f18948d19de96574a685bfda in term 1.
I20251028 03:00:04.924809 8550 log.cc:1199] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/46ae536d269d4cbdbcee44a26d25bd67
I20251028 03:00:04.925143 8550 ts_tablet_manager.cc:1950] T 46ae536d269d4cbdbcee44a26d25bd67 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.927542 8551 ts_tablet_manager.cc:1929] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:04.927704 8551 log.cc:1199] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/c21f794ec1284f5d99521954ed7ba9c9
I20251028 03:00:04.928049 8551 ts_tablet_manager.cc:1950] T c21f794ec1284f5d99521954ed7ba9c9 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.928805 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet c21f794ec1284f5d99521954ed7ba9c9 (table default.Table_2c98fdd4e565487c85c1d9128125e212 [id=575b84e0d3b743a0806edaae38121f92]) successfully deleted
I20251028 03:00:04.929111 8551 tablet_replica.cc:333] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.929293 8551 raft_consensus.cc:2243] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.930200 7965 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 9343ca8510de4eaa9945786e054a866d: Sending DeleteTablet for 3 replicas of tablet 900973d832c24b62a4554ae9a219cfd8
I20251028 03:00:04.930495 7965 catalog_manager.cc:6719] ReplaceTablet: tablet 900973d832c24b62a4554ae9a219cfd8 deleted and replaced by tablet 3e27552d470a405395dfcc7f9c1de92c
I20251028 03:00:04.931412 8256 tablet_service.cc:1552] Processing DeleteTablet for tablet 900973d832c24b62a4554ae9a219cfd8 with delete_type TABLET_DATA_DELETED (replaced by tablet 3e27552d470a405395dfcc7f9c1de92c) from {username='slave'} at 127.0.0.1:46256
I20251028 03:00:04.931588 8383 tablet_service.cc:1552] Processing DeleteTablet for tablet 900973d832c24b62a4554ae9a219cfd8 with delete_type TABLET_DATA_DELETED (replaced by tablet 3e27552d470a405395dfcc7f9c1de92c) from {username='slave'} at 127.0.0.1:57400
I20251028 03:00:04.931905 8551 raft_consensus.cc:2272] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.932386 8552 tablet_replica.cc:333] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: stopping tablet replica
I20251028 03:00:04.932554 8552 raft_consensus.cc:2243] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.932583 8551 ts_tablet_manager.cc:1916] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.932757 8552 raft_consensus.cc:2272] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.933163 8552 ts_tablet_manager.cc:1916] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.935019 8551 ts_tablet_manager.cc:1929] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:04.936352 8552 ts_tablet_manager.cc:1929] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20251028 03:00:04.936901 7965 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47364:
name: "default.Table_e99271148b364b8d9e4b49f25b23d77d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251028 03:00:04.937381 7965 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e99271148b364b8d9e4b49f25b23d77d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251028 03:00:04.938266 8551 log.cc:1199] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/583d6226a5f44d47a5ea4788c2b0d5ce
I20251028 03:00:04.938560 8551 ts_tablet_manager.cc:1950] T 583d6226a5f44d47a5ea4788c2b0d5ce P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.939811 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 583d6226a5f44d47a5ea4788c2b0d5ce (table default.Table_1e52c58d3e4042139159ba0e312a4089 [id=f5dcccbdb3e44a00a0169cff89823b0e]) successfully deleted
I20251028 03:00:04.940110 8551 tablet_replica.cc:333] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.940299 8551 raft_consensus.cc:2243] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.940548 8551 raft_consensus.cc:2272] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.941009 8551 ts_tablet_manager.cc:1916] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.935647 8113 tablet_service.cc:1552] Processing DeleteTablet for tablet 900973d832c24b62a4554ae9a219cfd8 with delete_type TABLET_DATA_DELETED (replaced by tablet 3e27552d470a405395dfcc7f9c1de92c) from {username='slave'} at 127.0.0.1:48434
I20251028 03:00:04.936816 10469 raft_consensus.cc:493] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.941963 10469 raft_consensus.cc:515] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.942328 10469 leader_election.cc:290] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.942703 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0ee59aeedd2c4c9ba94afe6578af649a" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.942821 8421 raft_consensus.cc:2468] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
I20251028 03:00:04.943145 8083 leader_election.cc:304] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:04.943406 10469 raft_consensus.cc:2804] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.943467 10469 raft_consensus.cc:493] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.943523 10469 raft_consensus.cc:3060] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.944276 10469 raft_consensus.cc:515] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.944623 10469 leader_election.cc:290] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525)
I20251028 03:00:04.943259 8551 ts_tablet_manager.cc:1929] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:04.945248 8551 log.cc:1199] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/338cf9d02142436f97ccd44313cc8199
I20251028 03:00:04.945415 8421 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0ee59aeedd2c4c9ba94afe6578af649a" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.945717 8421 raft_consensus.cc:3060] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.946707 8421 raft_consensus.cc:2468] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:04.946789 8544 tablet_bootstrap.cc:492] T d195958488bf401c8668116a97bf0f58 P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.947057 10546 raft_consensus.cc:493] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.947139 10546 raft_consensus.cc:515] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.947595 10546 leader_election.cc:290] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.947825 8083 leader_election.cc:304] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:04.947971 8544 tablet_bootstrap.cc:654] T d195958488bf401c8668116a97bf0f58 P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.948230 8551 ts_tablet_manager.cc:1950] T 338cf9d02142436f97ccd44313cc8199 P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
I20251028 03:00:04.948374 10468 raft_consensus.cc:2804] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.948482 10468 raft_consensus.cc:697] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:04.948619 10468 consensus_queue.cc:237] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.949076 7955 catalog_manager.cc:4985] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): tablet 338cf9d02142436f97ccd44313cc8199 (table default.Table_77a37bbdfa4f41458e6b50fb213850bc [id=5f61b3c12ed647968fc2932aa24a13f3]) successfully deleted
W20251028 03:00:04.949506 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 4b3047edf5a44514a23b689ee80ff1b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4b3047edf5a44514a23b689ee80ff1b4 already in progress: creating tablet
I20251028 03:00:04.949714 8146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72c8f9833434439da92711fe04e838af" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
W20251028 03:00:04.949915 7955 catalog_manager.cc:4965] TS d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): delete failed for tablet 7ee583e318174f34b162c2a104a0a3a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ee583e318174f34b162c2a104a0a3a9 already in progress: creating tablet
W20251028 03:00:04.950537 8354 leader_election.cc:343] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.951085 8551 tablet_replica.cc:333] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:04.951680 10468 consensus_queue.cc:1048] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251028 03:00:04.952615 8551 raft_consensus.cc:2243] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:04.952793 8551 raft_consensus.cc:2272] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.953343 8551 ts_tablet_manager.cc:1916] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.939033 8546 tablet_bootstrap.cc:492] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.954169 8552 log.cc:1199] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-1/wal/wals/900973d832c24b62a4554ae9a219cfd8
I20251028 03:00:04.943552 8274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0ee59aeedd2c4c9ba94afe6578af649a" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.954290 8546 ts_tablet_manager.cc:1403] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.000s
I20251028 03:00:04.954336 7786 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskJrsfcH/build/debug/bin/kudu with pid 7935
I20251028 03:00:04.954761 8546 raft_consensus.cc:359] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.954875 8546 raft_consensus.cc:385] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.954905 8546 raft_consensus.cc:740] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.955024 8546 consensus_queue.cc:260] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.955348 8546 ts_tablet_manager.cc:1434] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.955549 8546 tablet_bootstrap.cc:492] T 2fc2faa9c4f04fe3b931ce8a2808d738 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.956298 8550 tablet_replica.cc:333] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:04.956635 8546 tablet_bootstrap.cc:654] T 2fc2faa9c4f04fe3b931ce8a2808d738 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.956701 8550 raft_consensus.cc:2243] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:04.956878 8550 pending_rounds.cc:70] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
I20251028 03:00:04.956909 8550 pending_rounds.cc:77] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597175228624896 op_type: NO_OP noop_request { }
I20251028 03:00:04.956959 8550 raft_consensus.cc:2889] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:04.957006 8550 raft_consensus.cc:2272] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:04.945725 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0ee59aeedd2c4c9ba94afe6578af649a" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:04.957448 8552 ts_tablet_manager.cc:1950] T 900973d832c24b62a4554ae9a219cfd8 P ccbdf478f18948d19de96574a685bfda: Deleting consensus metadata
I20251028 03:00:04.948087 8283 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "72c8f9833434439da92711fe04e838af" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.950312 9958 raft_consensus.cc:493] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.958288 9958 raft_consensus.cc:515] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.958639 9958 leader_election.cc:290] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.951529 9595 raft_consensus.cc:493] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.959002 9595 raft_consensus.cc:515] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:04.951606 8553 raft_consensus.cc:493] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.959165 8553 raft_consensus.cc:515] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.952456 8273 raft_consensus.cc:3060] T f6818c3f2dfa48448bb2d9b38ed0dc19 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.959594 8553 leader_election.cc:290] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
W20251028 03:00:04.960853 8218 leader_election.cc:343] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: d5c3b02db24d4dc48260b42841844938
W20251028 03:00:04.961126 8220 leader_election.cc:343] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: d5c3b02db24d4dc48260b42841844938
W20251028 03:00:04.961181 8218 leader_election.cc:343] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Not found: Tablet not found: a4bcdc8a0c554b7cae81d7993e4f2861
W20251028 03:00:04.961839 8083 leader_election.cc:343] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
W20251028 03:00:04.962034 8083 leader_election.cc:343] T 0ee59aeedd2c4c9ba94afe6578af649a P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.961189 8220 leader_election.cc:304] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.962550 9958 raft_consensus.cc:2749] T d5c3b02db24d4dc48260b42841844938 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.962749 9595 leader_election.cc:290] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
W20251028 03:00:04.962914 8220 leader_election.cc:343] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: 59cd8e6357fe4a6cae6be04c493a6880
W20251028 03:00:04.963016 8220 leader_election.cc:343] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189): Not found: Tablet not found: a4bcdc8a0c554b7cae81d7993e4f2861
I20251028 03:00:04.963059 8220 leader_election.cc:304] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.963394 9595 raft_consensus.cc:2749] T a4bcdc8a0c554b7cae81d7993e4f2861 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.964013 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "59cd8e6357fe4a6cae6be04c493a6880" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.964355 8218 leader_election.cc:304] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccbdf478f18948d19de96574a685bfda; no voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.964582 8553 raft_consensus.cc:2749] T 59cd8e6357fe4a6cae6be04c493a6880 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.965001 8550 ts_tablet_manager.cc:1916] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:04.966207 8546 tablet_bootstrap.cc:492] T 2fc2faa9c4f04fe3b931ce8a2808d738 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.966343 8546 ts_tablet_manager.cc:1403] T 2fc2faa9c4f04fe3b931ce8a2808d738 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20251028 03:00:04.966897 8546 raft_consensus.cc:359] T 2fc2faa9c4f04fe3b931ce8a2808d738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.967054 8546 raft_consensus.cc:385] T 2fc2faa9c4f04fe3b931ce8a2808d738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.967128 8546 raft_consensus.cc:740] T 2fc2faa9c4f04fe3b931ce8a2808d738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.967271 8546 consensus_queue.cc:260] T 2fc2faa9c4f04fe3b931ce8a2808d738 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.967602 8550 ts_tablet_manager.cc:1929] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:04.967726 8546 ts_tablet_manager.cc:1434] T 2fc2faa9c4f04fe3b931ce8a2808d738 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.967679 8550 log.cc:1199] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ee1d686a0e4c47f6b214224047a55747
W20251028 03:00:04.960618 8356 leader_election.cc:343] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.968041 8550 ts_tablet_manager.cc:1950] T ee1d686a0e4c47f6b214224047a55747 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:04.968024 8356 leader_election.cc:304] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:04.968227 8546 tablet_bootstrap.cc:492] T 39f0a9883b584083b542d078ea7c165b P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.968294 10546 raft_consensus.cc:2749] T 72c8f9833434439da92711fe04e838af P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:04.969333 8546 tablet_bootstrap.cc:654] T 39f0a9883b584083b542d078ea7c165b P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.970463 8546 tablet_bootstrap.cc:492] T 39f0a9883b584083b542d078ea7c165b P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.970541 8546 ts_tablet_manager.cc:1403] T 39f0a9883b584083b542d078ea7c165b P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 03:00:04.970947 8546 raft_consensus.cc:359] T 39f0a9883b584083b542d078ea7c165b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.971045 8546 raft_consensus.cc:385] T 39f0a9883b584083b542d078ea7c165b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.971076 8546 raft_consensus.cc:740] T 39f0a9883b584083b542d078ea7c165b P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.971189 8546 consensus_queue.cc:260] T 39f0a9883b584083b542d078ea7c165b P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.971637 8546 ts_tablet_manager.cc:1434] T 39f0a9883b584083b542d078ea7c165b P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.972079 8546 tablet_bootstrap.cc:492] T d195958488bf401c8668116a97bf0f58 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.972106 8544 tablet_bootstrap.cc:492] T d195958488bf401c8668116a97bf0f58 P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.972184 8544 ts_tablet_manager.cc:1403] T d195958488bf401c8668116a97bf0f58 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20251028 03:00:04.972610 8544 raft_consensus.cc:359] T d195958488bf401c8668116a97bf0f58 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.972698 8544 raft_consensus.cc:385] T d195958488bf401c8668116a97bf0f58 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.972741 8544 raft_consensus.cc:740] T d195958488bf401c8668116a97bf0f58 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.972853 8544 consensus_queue.cc:260] T d195958488bf401c8668116a97bf0f58 P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.973111 8544 ts_tablet_manager.cc:1434] T d195958488bf401c8668116a97bf0f58 P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.973279 8544 tablet_bootstrap.cc:492] T 698cca4847e94a79933c644f6b1c1c8d P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.973351 8546 tablet_bootstrap.cc:654] T d195958488bf401c8668116a97bf0f58 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.974234 8544 tablet_bootstrap.cc:654] T 698cca4847e94a79933c644f6b1c1c8d P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.978457 10469 raft_consensus.cc:493] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.978552 10469 raft_consensus.cc:515] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.978892 10469 leader_election.cc:290] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.979449 8544 tablet_bootstrap.cc:492] T 698cca4847e94a79933c644f6b1c1c8d P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:04.979588 8544 ts_tablet_manager.cc:1403] T 698cca4847e94a79933c644f6b1c1c8d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20251028 03:00:04.979732 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37c267aaee7412b869753164556b9c0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.979882 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37c267aaee7412b869753164556b9c0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75" is_pre_election: true
I20251028 03:00:04.980005 8418 raft_consensus.cc:2468] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 0.
W20251028 03:00:04.980122 8083 leader_election.cc:343] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.980376 8083 leader_election.cc:304] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.980597 10469 raft_consensus.cc:2804] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.980649 10469 raft_consensus.cc:493] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.980687 10469 raft_consensus.cc:3060] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.980788 8544 raft_consensus.cc:359] T 698cca4847e94a79933c644f6b1c1c8d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.980888 8544 raft_consensus.cc:385] T 698cca4847e94a79933c644f6b1c1c8d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.980917 8544 raft_consensus.cc:740] T 698cca4847e94a79933c644f6b1c1c8d P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.981066 8544 consensus_queue.cc:260] T 698cca4847e94a79933c644f6b1c1c8d P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.981439 8544 ts_tablet_manager.cc:1434] T 698cca4847e94a79933c644f6b1c1c8d P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251028 03:00:04.981638 8544 tablet_bootstrap.cc:492] T 72c8f9833434439da92711fe04e838af P fbd2afb4894c4e7a8520cc09f8593aa0: Bootstrap starting.
I20251028 03:00:04.981690 10469 raft_consensus.cc:515] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.982014 10469 leader_election.cc:290] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189)
I20251028 03:00:04.982745 8418 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37c267aaee7412b869753164556b9c0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7e4cf694f71438298dd32a346abcf75"
I20251028 03:00:04.982873 8418 raft_consensus.cc:3060] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.982889 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d37c267aaee7412b869753164556b9c0" candidate_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
W20251028 03:00:04.983247 8083 leader_election.cc:343] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:04.983948 8418 raft_consensus.cc:2468] T d37c267aaee7412b869753164556b9c0 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd2afb4894c4e7a8520cc09f8593aa0 in term 1.
I20251028 03:00:04.984287 8083 leader_election.cc:304] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters: ccbdf478f18948d19de96574a685bfda
I20251028 03:00:04.984490 8544 tablet_bootstrap.cc:654] T 72c8f9833434439da92711fe04e838af P fbd2afb4894c4e7a8520cc09f8593aa0: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.984624 10469 raft_consensus.cc:2804] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.984717 10469 raft_consensus.cc:697] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Becoming Leader. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Running, Role: LEADER
I20251028 03:00:04.984870 10469 consensus_queue.cc:237] T d37c267aaee7412b869753164556b9c0 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.985616 8546 tablet_bootstrap.cc:492] T d195958488bf401c8668116a97bf0f58 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.985714 8546 ts_tablet_manager.cc:1403] T d195958488bf401c8668116a97bf0f58 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20251028 03:00:04.985729 10546 raft_consensus.cc:493] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:04.985821 10546 raft_consensus.cc:515] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.986184 10546 leader_election.cc:290] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.986274 8546 raft_consensus.cc:359] T d195958488bf401c8668116a97bf0f58 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.986382 8546 raft_consensus.cc:385] T d195958488bf401c8668116a97bf0f58 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.986418 8546 raft_consensus.cc:740] T d195958488bf401c8668116a97bf0f58 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.986524 8546 consensus_queue.cc:260] T d195958488bf401c8668116a97bf0f58 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.986678 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fa32a1433cd47a9ad39d1c613a18761" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:04.986788 8273 raft_consensus.cc:2468] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:04.986845 8546 ts_tablet_manager.cc:1434] T d195958488bf401c8668116a97bf0f58 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251028 03:00:04.987054 8546 tablet_bootstrap.cc:492] T 0ee59aeedd2c4c9ba94afe6578af649a P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.987170 8356 leader_election.cc:304] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:04.987677 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fa32a1433cd47a9ad39d1c613a18761" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:04.987789 8143 raft_consensus.cc:2468] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:04.988186 8546 tablet_bootstrap.cc:654] T 0ee59aeedd2c4c9ba94afe6578af649a P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.989569 8545 tablet_bootstrap.cc:492] T 746eae30ca284758a6e3ed8ad80c2cdc P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:04.989655 8545 ts_tablet_manager.cc:1403] T 746eae30ca284758a6e3ed8ad80c2cdc P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.082s user 0.002s sys 0.000s
I20251028 03:00:04.989672 10546 raft_consensus.cc:2804] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:04.989763 10546 raft_consensus.cc:493] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:04.989816 10546 raft_consensus.cc:3060] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.989881 8546 tablet_bootstrap.cc:492] T 0ee59aeedd2c4c9ba94afe6578af649a P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.989959 8546 ts_tablet_manager.cc:1403] T 0ee59aeedd2c4c9ba94afe6578af649a P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251028 03:00:04.990111 8545 raft_consensus.cc:359] T 746eae30ca284758a6e3ed8ad80c2cdc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.990203 8545 raft_consensus.cc:385] T 746eae30ca284758a6e3ed8ad80c2cdc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.990232 8545 raft_consensus.cc:740] T 746eae30ca284758a6e3ed8ad80c2cdc P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.990337 8545 consensus_queue.cc:260] T 746eae30ca284758a6e3ed8ad80c2cdc P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.990397 8546 raft_consensus.cc:359] T 0ee59aeedd2c4c9ba94afe6578af649a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.990495 8546 raft_consensus.cc:385] T 0ee59aeedd2c4c9ba94afe6578af649a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.990527 8546 raft_consensus.cc:740] T 0ee59aeedd2c4c9ba94afe6578af649a P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.990592 8545 ts_tablet_manager.cc:1434] T 746eae30ca284758a6e3ed8ad80c2cdc P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.990644 8546 consensus_queue.cc:260] T 0ee59aeedd2c4c9ba94afe6578af649a P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.990779 8545 tablet_bootstrap.cc:492] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:04.990945 8546 ts_tablet_manager.cc:1434] T 0ee59aeedd2c4c9ba94afe6578af649a P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.990923 10546 raft_consensus.cc:515] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.991146 8546 tablet_bootstrap.cc:492] T d37c267aaee7412b869753164556b9c0 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.991281 10546 leader_election.cc:290] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Requested vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:04.991789 8545 tablet_bootstrap.cc:654] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.992202 8546 tablet_bootstrap.cc:654] T d37c267aaee7412b869753164556b9c0 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.992941 8273 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fa32a1433cd47a9ad39d1c613a18761" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda"
I20251028 03:00:04.993053 8273 raft_consensus.cc:3060] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.993443 8546 tablet_bootstrap.cc:492] T d37c267aaee7412b869753164556b9c0 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.993520 8546 ts_tablet_manager.cc:1403] T d37c267aaee7412b869753164556b9c0 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 03:00:04.993997 8546 raft_consensus.cc:359] T d37c267aaee7412b869753164556b9c0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.994040 8273 raft_consensus.cc:2468] T 5fa32a1433cd47a9ad39d1c613a18761 P ccbdf478f18948d19de96574a685bfda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:04.994117 8546 raft_consensus.cc:385] T d37c267aaee7412b869753164556b9c0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.994148 8546 raft_consensus.cc:740] T d37c267aaee7412b869753164556b9c0 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.994251 8546 consensus_queue.cc:260] T d37c267aaee7412b869753164556b9c0 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:04.994517 8546 ts_tablet_manager.cc:1434] T d37c267aaee7412b869753164556b9c0 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.994784 8546 tablet_bootstrap.cc:492] T c20410feb13543db9642c1e40bfb9300 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.994808 8356 leader_election.cc:304] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccbdf478f18948d19de96574a685bfda, d7e4cf694f71438298dd32a346abcf75; no voters:
I20251028 03:00:04.994964 10242 raft_consensus.cc:2804] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Leader election won for term 1
I20251028 03:00:04.995043 10242 raft_consensus.cc:697] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Becoming Leader. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Running, Role: LEADER
I20251028 03:00:04.995174 10242 consensus_queue.cc:237] T 5fa32a1433cd47a9ad39d1c613a18761 P d7e4cf694f71438298dd32a346abcf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.995987 8546 tablet_bootstrap.cc:654] T c20410feb13543db9642c1e40bfb9300 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:04.997069 8546 tablet_bootstrap.cc:492] T c20410feb13543db9642c1e40bfb9300 P ccbdf478f18948d19de96574a685bfda: No bootstrap required, opened a new log
I20251028 03:00:04.997140 8546 ts_tablet_manager.cc:1403] T c20410feb13543db9642c1e40bfb9300 P ccbdf478f18948d19de96574a685bfda: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20251028 03:00:04.997316 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5fa32a1433cd47a9ad39d1c613a18761" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0"
I20251028 03:00:04.997434 8143 raft_consensus.cc:3060] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:04.997572 8546 raft_consensus.cc:359] T c20410feb13543db9642c1e40bfb9300 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.997668 8546 raft_consensus.cc:385] T c20410feb13543db9642c1e40bfb9300 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:04.997701 8546 raft_consensus.cc:740] T c20410feb13543db9642c1e40bfb9300 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbdf478f18948d19de96574a685bfda, State: Initialized, Role: FOLLOWER
I20251028 03:00:04.997819 8546 consensus_queue.cc:260] T c20410feb13543db9642c1e40bfb9300 P ccbdf478f18948d19de96574a685bfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:04.998186 8546 ts_tablet_manager.cc:1434] T c20410feb13543db9642c1e40bfb9300 P ccbdf478f18948d19de96574a685bfda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:04.998371 8546 tablet_bootstrap.cc:492] T 4393587f022446828e344a9c394a92f9 P ccbdf478f18948d19de96574a685bfda: Bootstrap starting.
I20251028 03:00:04.998438 8143 raft_consensus.cc:2468] T 5fa32a1433cd47a9ad39d1c613a18761 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 1.
I20251028 03:00:04.999413 8546 tablet_bootstrap.cc:654] T 4393587f022446828e344a9c394a92f9 P ccbdf478f18948d19de96574a685bfda: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:05.008574 8550 tablet_replica.cc:333] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:05.008785 8550 raft_consensus.cc:2243] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:05.008919 8550 raft_consensus.cc:2272] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:05.009198 8219 connection.cc:537] server connection from 127.0.0.1:46256 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251028 03:00:05.009337 8551 ts_tablet_manager.cc:1929] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251028 03:00:05.009375 8550 ts_tablet_manager.cc:1916] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:05.009419 8551 log.cc:1199] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-2/wal/wals/0ee59aeedd2c4c9ba94afe6578af649a
I20251028 03:00:05.009649 8551 ts_tablet_manager.cc:1950] T 0ee59aeedd2c4c9ba94afe6578af649a P d7e4cf694f71438298dd32a346abcf75: Deleting consensus metadata
W20251028 03:00:05.009851 8475 connection.cc:537] client connection to 127.7.154.188:44237 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251028 03:00:05.010421 8475 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.154.188:44237)
W20251028 03:00:05.010591 8475 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.7.154.188:44237)
W20251028 03:00:05.010699 8475 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.7.154.188:44237)
W20251028 03:00:05.010967 8475 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.7.154.188:44237)
I20251028 03:00:05.011193 8551 tablet_replica.cc:333] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75: stopping tablet replica
I20251028 03:00:05.011355 8551 raft_consensus.cc:2243] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:05.011569 8551 pending_rounds.cc:70] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75: Trying to abort 1 pending ops.
I20251028 03:00:05.011601 8551 pending_rounds.cc:77] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597177397927936 op_type: NO_OP noop_request { }
I20251028 03:00:05.011662 8551 raft_consensus.cc:2889] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:05.011710 8551 raft_consensus.cc:2272] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:05.012727 8551 ts_tablet_manager.cc:1916] T 39f0a9883b584083b542d078ea7c165b P d7e4cf694f71438298dd32a346abcf75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:05.015372 8273 raft_consensus.cc:3060] T 2fc2faa9c4f04fe3b931ce8a2808d738 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:05.016570 8545 tablet_bootstrap.cc:492] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:05.016652 8545 ts_tablet_manager.cc:1403] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.002s
I20251028 03:00:05.017122 8545 raft_consensus.cc:359] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:05.017215 8545 raft_consensus.cc:385] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:05.017246 8545 raft_consensus.cc:740] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:05.017351 8545 consensus_queue.cc:260] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:05.017604 8545 ts_tablet_manager.cc:1434] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251028 03:00:05.019575 8550 ts_tablet_manager.cc:1929] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:05.019740 8550 log.cc:1199] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/ac782a9f0b564667a7b20f1e328f2572
I20251028 03:00:05.020092 8550 ts_tablet_manager.cc:1950] T ac782a9f0b564667a7b20f1e328f2572 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 03:00:05.020270 8356 connection.cc:537] server connection from 127.0.0.1:57400 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251028 03:00:05.020509 8083 connection.cc:537] server connection from 127.0.0.1:48434 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251028 03:00:05.021602 10469 consensus_queue.cc:1048] T 2fc2faa9c4f04fe3b931ce8a2808d738 P fbd2afb4894c4e7a8520cc09f8593aa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251028 03:00:05.021657 8083 connection.cc:441] server connection from 127.0.0.1:48434 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:48434 (request call id 1229) could send its response
W20251028 03:00:05.021835 10582 negotiation.cc:337] Failed RPC negotiation. Trace:
1028 03:00:05.015476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.154.188:44237 (local address 127.7.154.131:33471)
1028 03:00:05.021654 (+ 6178us) negotiation.cc:107] Waiting for socket to connect
1028 03:00:05.021716 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.154.188:44237: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":5993,"thread_start_us":71,"threads_started":1}
W20251028 03:00:05.022291 8083 connection.cc:441] server connection from 127.0.0.1:48434 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:48434 (request call id 1232) could send its response
I20251028 03:00:05.022426 8550 tablet_replica.cc:333] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:05.022563 8550 raft_consensus.cc:2243] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251028 03:00:05.022677 8550 raft_consensus.cc:2272] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251028 03:00:05.023072 8550 ts_tablet_manager.cc:1916] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251028 03:00:05.025635 8550 ts_tablet_manager.cc:1929] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251028 03:00:05.025766 8550 log.cc:1199] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/c21f794ec1284f5d99521954ed7ba9c9
I20251028 03:00:05.022079 8545 tablet_bootstrap.cc:492] T 4eaa472029a94052abb49241b7622d0f P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:05.026117 8550 ts_tablet_manager.cc:1950] T c21f794ec1284f5d99521954ed7ba9c9 P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
W20251028 03:00:05.026711 8083 connection.cc:441] server connection from 127.0.0.1:48434 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:48434 (request call id 1233) could send its response
I20251028 03:00:05.027212 8545 tablet_bootstrap.cc:654] T 4eaa472029a94052abb49241b7622d0f P d7e4cf694f71438298dd32a346abcf75: Neither blocks nor log segments found. Creating new log.
I20251028 03:00:05.025313 10546 raft_consensus.cc:493] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:05.027745 10546 raft_consensus.cc:515] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:05.028132 10546 leader_election.cc:290] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:05.028441 8550 tablet_replica.cc:333] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: stopping tablet replica
I20251028 03:00:05.028582 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2d0e84daebfe481c9df8459e33b46e16" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:05.028623 8550 raft_consensus.cc:2243] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: Raft consensus shutting down.
I20251028 03:00:05.028729 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2d0e84daebfe481c9df8459e33b46e16" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
W20251028 03:00:05.028937 8354 leader_election.cc:343] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:05.029119 8550 pending_rounds.cc:70] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: Trying to abort 1 pending ops.
W20251028 03:00:05.029111 8356 leader_election.cc:343] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Illegal state: must be running to vote when last-logged opid is not known
I20251028 03:00:05.029175 8356 leader_election.cc:304] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75; no voters: ccbdf478f18948d19de96574a685bfda, fbd2afb4894c4e7a8520cc09f8593aa0
I20251028 03:00:05.029263 8550 pending_rounds.cc:77] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7215597176836325376 op_type: NO_OP noop_request { }
I20251028 03:00:05.029361 8550 raft_consensus.cc:2889] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251028 03:00:05.029434 10546 raft_consensus.cc:2749] T 2d0e84daebfe481c9df8459e33b46e16 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251028 03:00:05.029451 8550 raft_consensus.cc:2272] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251028 03:00:05.031214 8356 consensus_peers.cc:597] T 6bd72bb34fe843079f8a4b828770e441 P d7e4cf694f71438298dd32a346abcf75 -> Peer ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525): Couldn't send request to peer ccbdf478f18948d19de96574a685bfda. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6bd72bb34fe843079f8a4b828770e441. This is attempt 1: this message will repeat every 5th retry.
W20251028 03:00:05.031668 10583 negotiation.cc:337] Failed RPC negotiation. Trace:
1028 03:00:05.020892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.154.188:44237 (local address 127.7.154.130:57677)
1028 03:00:05.031469 (+ 10577us) negotiation.cc:107] Waiting for socket to connect
1028 03:00:05.031531 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.154.188:44237: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":10496,"thread_start_us":101,"threads_started":1}
I20251028 03:00:05.033911 8550 ts_tablet_manager.cc:1916] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20251028 03:00:05.034000 10579 negotiation.cc:337] Failed RPC negotiation. Trace:
1028 03:00:05.011687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.154.188:44237 (local address 127.0.0.1:57052)
1028 03:00:05.023096 (+ 11409us) negotiation.cc:107] Waiting for socket to connect
1028 03:00:05.023153 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.154.188:44237: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":10542,"spinlock_wait_cycles":1806208,"thread_start_us":95,"threads_started":1}
I20251028 03:00:05.035090 10245 raft_consensus.cc:493] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:05.035203 10245 raft_consensus.cc:515] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:05.035647 10245 leader_election.cc:290] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbdf478f18948d19de96574a685bfda (127.7.154.130:43525), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:05.035653 8553 raft_consensus.cc:493] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251028 03:00:05.035742 8553 raft_consensus.cc:515] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } }
I20251028 03:00:05.036046 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c20410feb13543db9642c1e40bfb9300" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:05.036123 8553 leader_election.cc:290] T 583d6226a5f44d47a5ea4788c2b0d5ce P ccbdf478f18948d19de96574a685bfda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7e4cf694f71438298dd32a346abcf75 (127.7.154.131:33189), fbd2afb4894c4e7a8520cc09f8593aa0 (127.7.154.129:46529)
I20251028 03:00:05.036170 8143 raft_consensus.cc:2468] T c20410feb13543db9642c1e40bfb9300 P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:05.036264 8270 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c20410feb13543db9642c1e40bfb9300" candidate_uuid: "d7e4cf694f71438298dd32a346abcf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbdf478f18948d19de96574a685bfda" is_pre_election: true
I20251028 03:00:05.036370 8545 tablet_bootstrap.cc:492] T 4eaa472029a94052abb49241b7622d0f P d7e4cf694f71438298dd32a346abcf75: No bootstrap required, opened a new log
I20251028 03:00:05.036360 8270 raft_consensus.cc:2468] T c20410feb13543db9642c1e40bfb9300 P ccbdf478f18948d19de96574a685bfda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e4cf694f71438298dd32a346abcf75 in term 0.
I20251028 03:00:05.036463 8354 leader_election.cc:304] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7e4cf694f71438298dd32a346abcf75, fbd2afb4894c4e7a8520cc09f8593aa0; no voters:
I20251028 03:00:05.036470 8545 ts_tablet_manager.cc:1403] T 4eaa472029a94052abb49241b7622d0f P d7e4cf694f71438298dd32a346abcf75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20251028 03:00:05.037029 8143 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "583d6226a5f44d47a5ea4788c2b0d5ce" candidate_uuid: "ccbdf478f18948d19de96574a685bfda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" is_pre_election: true
I20251028 03:00:05.037070 8545 raft_consensus.cc:359] T 4eaa472029a94052abb49241b7622d0f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:05.037154 8545 raft_consensus.cc:385] T 4eaa472029a94052abb49241b7622d0f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:05.037185 8545 raft_consensus.cc:740] T 4eaa472029a94052abb49241b7622d0f P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4cf694f71438298dd32a346abcf75, State: Initialized, Role: FOLLOWER
I20251028 03:00:05.037312 8545 consensus_queue.cc:260] T 4eaa472029a94052abb49241b7622d0f P d7e4cf694f71438298dd32a346abcf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } }
I20251028 03:00:05.037479 10546 raft_consensus.cc:2804] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251028 03:00:05.037526 10546 raft_consensus.cc:493] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251028 03:00:05.037534 8544 tablet_bootstrap.cc:492] T 72c8f9833434439da92711fe04e838af P fbd2afb4894c4e7a8520cc09f8593aa0: No bootstrap required, opened a new log
I20251028 03:00:05.037565 10546 raft_consensus.cc:3060] T c20410feb13543db9642c1e40bfb9300 P d7e4cf694f71438298dd32a346abcf75 [term 0 FOLLOWER]: Advancing to term 1
I20251028 03:00:05.037619 8544 ts_tablet_manager.cc:1403] T 72c8f9833434439da92711fe04e838af P fbd2afb4894c4e7a8520cc09f8593aa0: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.000s
I20251028 03:00:05.037637 8550 ts_tablet_manager.cc:1929] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251028 03:00:05.037643 8545 ts_tablet_manager.cc:1434] T 4eaa472029a94052abb49241b7622d0f P d7e4cf694f71438298dd32a346abcf75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251028 03:00:05.037688 8550 log.cc:1199] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting WAL directory at /tmp/dist-test-taskJrsfcH/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1761620388201885-7786-0/minicluster-data/ts-0/wal/wals/583d6226a5f44d47a5ea4788c2b0d5ce
I20251028 03:00:05.037861 8545 tablet_bootstrap.cc:492] T 02f1a2d489d74e33ab991611fd55966d P d7e4cf694f71438298dd32a346abcf75: Bootstrap starting.
I20251028 03:00:05.038100 8550 ts_tablet_manager.cc:1950] T 583d6226a5f44d47a5ea4788c2b0d5ce P fbd2afb4894c4e7a8520cc09f8593aa0: Deleting consensus metadata
I20251028 03:00:05.038182 8544 raft_consensus.cc:359] T 72c8f9833434439da92711fe04e838af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129" port: 46529 } }
I20251028 03:00:05.038273 8544 raft_consensus.cc:385] T 72c8f9833434439da92711fe04e838af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251028 03:00:05.038303 8544 raft_consensus.cc:740] T 72c8f9833434439da92711fe04e838af P fbd2afb4894c4e7a8520cc09f8593aa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd2afb4894c4e7a8520cc09f8593aa0, State: Initialized, Role: FOLLOWER
I20251028 03:00:05.038411 8544 consensus_queue.cc:260] T 72c8f9833434439da92711fe04e838af P fbd2afb4894c4e7a8520cc09f8593aa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e4cf694f71438298dd32a346abcf75" member_type: VOTER last_known_addr { host: "127.7.154.131" port: 33189 } } peers { permanent_uuid: "ccbdf478f18948d19de96574a685bfda" member_type: VOTER last_known_addr { host: "127.7.154.130" port: 43525 } } peers { permanent_uuid: "fbd2afb4894c4e7a8520cc09f8593aa0" member_type: VOTER last_known_addr { host: "127.7.154.129"